systems engineering models for signature injuries of ...1483/fulltext.pdf · systems engineering...
TRANSCRIPT
SYSTEMS ENGINEERING MODELS FOR SIGNATURE INJURIES OF MODERN MILITARY CONFLICTS
A Dissertation Presented
by
Hande Muşdal
to
The Department of Mechanical and Industrial Engineering
in partial fulfillment of the requirements for the degree of
Doctor of Philosophy
in the field of
Industrial Engineering
Northeastern University Boston, Massachusetts
August 2013
©Copyright 2013 by Hande Muşdal and James C. Benneyan All Rights Reserved
iii
Abstract
Modern military conflicts are producing dramatic increases in a new class of “silent” injuries, in
part due to the changing manners by which war is waged and better protective equipment.
Foremost among these are traumatic brain injury (TBI), posttraumatic stress disorder (PTSD),
depression, and various mental health issues. The estimated prevalence of TBI is significantly
higher in the military when compared to the general population, with the vast majority of soldiers
being diagnosed with mild TBI which occurs with no outward signs of trauma. PTSD, on the
other hand, is an increasingly important problem among U.S. service members and one of the
signature injuries in the Iraq and Afghanistan wars. Since associated problems with these
disorders often are cognitive, emotional, and behavioral, many cases go undetected and untreated
indefinitely, linked with significant psychological disorders, long-term disabilities, and economic
burdens.
This dissertation presents several systems engineering models to optimize the overall design,
effectiveness, and capacity of healthcare systems for detecting and treating silent injuries, such
as TBI and PTSD, as well as a general health problem that is common among veterans, sleep
apnea, by addressing the following needs: (1) sequential screening processes, (2) categorical
diagnostic methods, and (3) care services location-allocation (network optimization) models.
iv
The first focus of this dissertation is analyzing and optimizing the design of disease screening
processes. Several probability and Monte Carlo simulation models are developed to investigate
the current and proposed PTSD screening processes within the Veterans Health Administration
(VHA). Results indicate that a more systematically designed system, which consists of a series of
annual screenings along with a standardized confirmatory testing, results in lower false diagnosis
rates, predictable performance, and reduced costs. Additionally, a sequential screening process
for mild TBI is proposed and illustrated in order to give an insight into how the general approach
is applied to other disorders.
The second area of focus is developing multi-state categorical diagnostic models, which combine
different assessment tools and consist of multiple screenings over time, in order to improve the
diagnosis reliability and accuracy. Three types of predictive models – fuzzy logic, logistic
regression, and neural networks – are described and used to determine whether or not an
individual has TBI and to categorize him into the most likely severity state. A numerical example
is given for illustration purposes and results indicate that these models can help reduce the
number of unrecognized and misdiagnosed cases.
Finally, the third part of this dissertation illustrates the use of location-allocation models in order
to improve access to care and patient satisfaction while minimizing overall system cost. Several
single and multi-period integer programming models are developed and used across a range of
specialty care services, namely, PTSD treatment and sleep apnea testing services, within the
VHA. Results indicate significant opportunity to simultaneously reduce total cost, reduce total
travel distances, and increase within-network access.
v
Acknowledgments
To my mom and dad for their unconditional love and endless belief in me…
The pages of this dissertation hold far more than the conclusion of years of study; they also
reflect the relationships with many great people to whom I owe my deepest gratitude for being a
part of this journey.
First and foremost, I would like to express my sincere appreciation to my adviser, Dr. James C.
Benneyan, for the tremendous support and opportunities he provided throughout my doctoral
studies. Without his continuous guidance, encouragement, patience, and knowledge, this
dissertation would not have been possible. His vision, wisdom, and expertise have made him a
constant oasis of ideas, which has exceptionally inspired and enriched my growth as a student
and researcher. I consider myself amazingly fortunate to have had such a great mentor.
I would like to thank Dr. Brian Shiner and Dr. Thomas P. Cullinane for accepting to be a part of
my dissertation committee. I truly appreciate their constructive comments, time, and support.
Their invaluable contribution made this dissertation a complete work. I also extend my
appreciation to Dr. Bradley V. Watts for his time and feedback.
vi
A special appreciation goes to my project teammate and good friend, Seda Sinangil, whose
contribution to a part of this dissertation is indisputable. Her hard work and colorful personality
have been a great source of motivation and joy for me even during the most challenging times.
I am truly thankful to my ex-office mates but forever friends, Serpil Mutlu and Zeynep Karakuş,
for their invaluable friendship and support; it was their company that made this journey
enjoyable. I would also like to thank all my other ex-office mates for the joy they brought to my
life. Each and every one of them has great places in my heart.
I would like to thank Zeynep Damla and Çağdaş Önal for being the perfect company during our
long Friday night dinners and weekend trips, making Boston a home away from home, and being
a family away from family. My sincere thanks also go to my dear friends Aysun Taşeli, Aysun
Sünnetçi, and Mehmet Erkan Ceyhan for enriching my life in so many ways.
I am deeply grateful to my mother-in-law, Sefanur Öndemir, and my father-in-law, Mehmet
Öndemir, for loving and supporting me as their daughter. I would like to thank Aykut Öndemir,
Mine Öndemir, Volkan Öndemir, and Dilek Öndemir for welcoming me warmly into their
family. I am also thankful to Gözde Öndemir, Nazlı Öndemir, and Duru Öndemir for making my
life more delightful.
My greatest gratitude goes to my family for shaping me into who I am today, loving me
unconditionally, and believing in me no matter what. I thank my mother, İnci Muşdal, for taking
care of me even across the other side of the world. I thank my father, Yılmaz Muşdal, for
supporting every single decision that I made. I thank my brother, Uygar Muşdal, for always
vii
being there for me and cheering me up even in the darkest times. He is the other part of my soul,
the one I have fought the most, and the one I have loved the most. I am also thankful to my aunt,
Günseli Gürbulak, and her husband, Talat Gürbulak, for caring for me no less than their own
children and to my lovely cousins, Sezin Gürbulak and Ulaş Gürbulak, for making my life more
beautiful every single day.
Last but not least, I am wholeheartedly grateful to my husband, Önder Öndemir, for his endless
patience, dedicated support, and unwavering love. Since our first day together, he has been my
constant source of comfort, joy, and inspiration, the best friend I could ever ask for, and the
reason to look forward to each new day. I cannot thank him enough for standing by me always,
during my best and even my worst.
A part of this dissertation was funded by the Center for Health Organization Transformation
(National Science Foundation grant IIP-10341990) and the VISN 1 Veterans Engineering
Resource Center (grant VA 241-P-1772).
viii
Table of Contents
Abstract .......................................................................................................................................... iii
Acknowledgments........................................................................................................................... v
Table of Contents ......................................................................................................................... viii
List of Figures ............................................................................................................................... xii
List of Tables .............................................................................................................................. xvii
List of Abbreviations ................................................................................................................... xix
Introduction ................................................................................................................ 1 Chapter 1 -
1.1 Silent Injuries ...................................................................................................................... 1
1.1.1 Traumatic Brain Injury ................................................................................................ 2
1.1.2 Posttraumatic Stress Disorder ...................................................................................... 6
1.2 Sleep Apnea ...................................................................................................................... 10
1.3 Motivation for Work ......................................................................................................... 14
Sequential Screening Models ................................................................................... 17 Chapter 2 -
2.1 Background ....................................................................................................................... 17
2.2 PTSD Screening Process within the VHA ........................................................................ 19
2.3 Methodology ..................................................................................................................... 23
2.3.1 Notation and Assumptions ......................................................................................... 23
ix
2.3.2 Overall System Sensitivity ........................................................................................ 28
2.3.2.1 Probabilities of Detecting Truly Positive Patients and of a False-Negative .................................... 28
2.3.2.2 Expected Number and Distribution of False-Negatives .................................................................. 30
2.3.3 Overall System Specificity ........................................................................................ 32
2.3.3.1 Probabilities of Detecting Truly Negative Patients and of a False-Positive .................................... 32
2.3.3.2 Expected Number and Distribution of False-Positives ................................................................... 34
2.3.4 Expected Cost ............................................................................................................ 35
2.4 Scenario Analysis.............................................................................................................. 38
2.4.1 Effect of Scoring Methods on Overall System Accuracy, Cost, and Workload ........ 38
2.4.2 Effect of False Diagnosis Costs on the Total Cost of Screening Protocols ............... 49
2.4.3 Effect of Compliance Rates on Overall System Accuracy and Cost ......................... 49
2.4.4 Effect of PTSD Prevalence on the Predictive Value of Screening Protocols ............ 52
2.5 Other Possible Applications: A Sequential Screening Model for Mild TBI .................... 53
2.6 Discussion ......................................................................................................................... 59
Multi-State Categorical Diagnostic Models ............................................................. 61 Chapter 3 -
3.1 Background ....................................................................................................................... 61
3.2 Fuzzy Categorical Model .................................................................................................. 64
3.2.1 Methodology .............................................................................................................. 64
3.2.2 Numerical Example ................................................................................................... 68
3.2.3 Model Accuracy ......................................................................................................... 71
3.3 Multinomial Logistic Regression Categorical Model ....................................................... 73
3.3.1 Methodology .............................................................................................................. 73
3.3.2 Numerical Example ................................................................................................... 75
3.3.3 Model Accuracy ......................................................................................................... 76
x
3.4 Artificial Neural Network Model...................................................................................... 78
3.4.1 Methodology .............................................................................................................. 78
3.4.2 Numerical Example ................................................................................................... 81
3.4.3 Model Accuracy ......................................................................................................... 81
3.5 Model Comparison and Optimization ............................................................................... 82
3.6 Discussion ......................................................................................................................... 84
Network Optimization Models ................................................................................. 86 Chapter 4 -
4.1 Background ....................................................................................................................... 86
4.2 Optimal Location and Use of In-Person and Video-Based PTSD Treatment Services
within the VHA ......................................................................................................................... 90
4.2.1 VA New England PTSD Treatment Services ............................................................ 91
4.2.2 Methodology .............................................................................................................. 95
4.2.2.1 Model Overview and Assumptions ................................................................................................. 95
4.2.2.2 Integer Programming Model ........................................................................................................... 97
4.2.3 Application and Results ............................................................................................. 99
4.2.3.1 Application Details ......................................................................................................................... 99
4.2.3.2 Results ............................................................................................................................................. 99
4.2.4 Discussion ................................................................................................................ 104
4.3 Single and Multi-Period Location-Allocation Models for Sleep Apnea Testing Services
within the VHA ....................................................................................................................... 106
4.3.1 VA New England Sleep Apnea Services ................................................................. 106
4.3.2 Methodology ............................................................................................................ 112
4.3.2.1 Model Overview and Assumptions ............................................................................................... 112
4.3.2.2 Single-Period Model ..................................................................................................................... 113
4.3.2.3 Multi-Period Model....................................................................................................................... 116
xi
4.3.3 Application and Results ........................................................................................... 118
4.3.3.1 Application Details ....................................................................................................................... 118
4.3.3.2 Single-Period Results .................................................................................................................... 120
4.3.3.3 Multi-Period Results ..................................................................................................................... 126
4.3.4 Sensitivity Analysis on Demand Variability ........................................................... 132
4.3.5 Discussion ................................................................................................................ 136
Conclusions and Future Work ................................................................................ 138 Chapter 5 -
Bibliography ............................................................................................................................... 143
Appendix A - Overall Sensitivity of PTSD Screening Process ................................................. 163
Appendix B - Overall Specificity of PTSD Screening Process ................................................. 167
Appendix C - Expected Total Cost of PTSD Screening Process ............................................... 170
Appendix D - Overall Accuracy and Cost of TBI Screening Process ....................................... 179
xii
List of Figures
Figure 2-1 Current PTSD screening process in the VA ................................................................ 21
Figure 2-2 Proposed PTSD screening process in the VA ............................................................. 22
Figure 2-3 Probability distribution of cost of (a) a false-positive and (b) a false-negative result 26
Figure 2-4 Probability distributions of number of false diagnoses per 10,000 patients in the first
year ........................................................................................................................................ 31
Figure 2-5 Probability distribution of total cost per 10,000 patients (obtained via simulation) ... 37
Figure 2-6 Comparison of past, current, and proposed screening protocols for PTSD
identification in the VA ......................................................................................................... 40
Figure 2-7 Effect of scoring method on overall (a) sensitivity and (b) specificity over a 20-year
screening horizon ................................................................................................................... 41
Figure 2-8 Effect of scoring method on expected number of (a) false-negatives and (b) false-
positives over a 20-year screening horizon ........................................................................... 42
Figure 2-9 Effect of scoring method on expected (a) screening cost and (b) total cost over a 20-
year screening horizon ........................................................................................................... 43
xiii
Figure 2-10 Decomposition of expected (a) total cost and (b) amount of screening tests assuming
protocol 1 ............................................................................................................................... 44
Figure 2-11 Tradeoff between stopping criteria and expected (a) number of false diagnoses, and
(b) cost at the end of screening horizon (j = 20) assuming protocol 1 .................................. 45
Figure 2-12 Effect of discontinued screening after three negative results on expected (a) number
of false-negatives and (b) number of false-positives assuming protocol 1 ........................... 46
Figure 2-13 Effect of discontinued screening after three negative results on expected (a)
screening cost and (b) total cost assuming protocol 1 ........................................................... 47
Figure 2-14 Effect of false-negative/false-positive cost ratio on total cost assuming protocol 1
with various stopping criteria ................................................................................................ 49
Figure 2-15 Effect of compliance rates on expected (a) number of false-negatives and (b) number
of false-positives assuming protocol 1 with discontinued screening after three negative
results ..................................................................................................................................... 51
Figure 2-16 Effect of compliance rates on expected (a) screening cost and (b) total cost assuming
protocol 1 with discontinued screening after three negative results ...................................... 52
Figure 2-17 Effect of PTSD prevalence on predictive value of protocol 1 with discontinued
screening after three negative results ..................................................................................... 53
Figure 2-18 A sequential screening process for mild TBI ............................................................ 55
Figure 2-19 Competing effect of number of screening tests on overall sensitivity versus
specificity, and the role of convergence to bounds, where (a) both system bounds = 0.95 and
(b) both system bounds = 0.80 ............................................................................................... 57
xiv
Figure 2-20 Comparison of expected total cost per 100 patients .................................................. 58
Figure 3-1 General illustration of longitudinal TBI classification model..................................... 64
Figure 3-2 Fuzzy membership functions for trauma severity degrees .......................................... 66
Figure 3-3 Accuracy of fuzzy categorical model results over time .............................................. 72
Figure 3-4 A graphical comparison of link functions ................................................................... 74
Figure 3-5 Accuracy of MLR categorical model results over time .............................................. 77
Figure 3-6 Generic structure of an ANN ...................................................................................... 79
Figure 3-7 Illustration of general input-output neuron activity .................................................... 80
Figure 3-8 ANN network established in Neurosolutions for N = 4 screening test example ......... 81
Figure 3-9 Accuracy of ANN categorical model results over time .............................................. 82
Figure 4-1 Estimated PTSD care demand by state across VISN 1 (January 2009 - May 2010) .. 93
Figure 4-2 Geographic distribution of veterans with PTSD in (a) raw counts and (b) per square
mile by 3-digit zip code and all VA facilities across VISN 1 (January 2009 - May 2010) ... 94
Figure 4-3 VA outpatient service use among patients with a primary PTSD diagnosis across
VISN 1 (October 2009 - September 2010) ............................................................................ 95
Figure 4-4 Tradeoff between acceptable distance and (a) total annual cost, and (b) coverage
percentage assuming estimated care demand in 2010 ......................................................... 100
Figure 4-5 Optimal areas for video-based services for a (a) 15-mile, (b) 30-mile, (c) 45-mile, and
(d) 60-mile maximum acceptable distance assuming estimated care demand in 2010 ....... 103
Figure 4-6 Estimates of future PTSD care needs over the next 10 years (2010 - 2020) ............ 104
xv
Figure 4-7 Sleep apnea care demand by state across VISN 1 (January 2009 - May 2010) ........ 107
Figure 4-8 Estimated sleep apnea care demand by 3-digit zip code and all VA facilities across
VISN 1 (January 2009 - May 2010) .................................................................................... 108
Figure 4-9 Fee visits for West Haven, (a) number of fee visits versus monthly utilization, (b)
weekly fee visits versus in-house visits ............................................................................... 110
Figure 4-10 Utilization of sleep apnea capacity by medical center over time ............................ 111
Figure 4-11 Travel distance distribution, VISN 1 (January 2009 - May 2010) .......................... 111
Figure 4-12 Total projected sleep apnea demand by year for entire VISN 1 (New England) .... 118
Figure 4-13 Optimized tradeoff between acceptable distance and (a) total cost, and (b) coverage
percentage for single-period model assuming current observed demands and case 3 ........ 121
Figure 4-14 Optimized tradeoff between acceptable distance and (a) total cost, and (b) coverage
percentage for single-period model assuming estimated true demand and candidate facilities
under case 3 ......................................................................................................................... 123
Figure 4-15 Demand nodes covered within-network and outside network and VA facilities that
provide service for single-period model assuming a 60-mile acceptable maximum distance
and candidate facilities under case 3 ................................................................................... 126
Figure 4-16 Optimized tradeoff between acceptable distance and (a) average annual cost, and (b)
coverage percentage for multi-period model assuming case 3 ............................................ 128
Figure 4-17 Optimal (a) annual cost and (b) coverage percentages considering different planning
horizons (both for case 3 and P = 3) .................................................................................... 131
Figure 4-18 Pseudo code for the simulation models with demand variability ........................... 132
xvi
Figure 4-19 Changes in average (a) total cost, (b) number of unanticipated fee patients, (c) house
cost, and (d) fee cost assuming variable annual demands and single period model results for
case 3 and P = 3 ................................................................................................................... 134
Figure 4-20 Changes in average (a) total cost, (b) number of unanticipated fee patients, (c) house
cost, and (d) fee cost assuming variable monthly demands and single period model results
for case 3 and P = 3 ............................................................................................................. 135
xvii
List of Tables
Table 2-1 Main characteristics of the PC-PTSD, PCL, and CAPS screening tests ...................... 22
Table 2-2 Minimum, average, and maximum cost estimates for a false-positive and false-
negative result under the assumption of different cases ........................................................ 26
Table 2-3 Summary of analytic distributions of cost, workload, and accuracy with comparison to
simulation results ................................................................................................................... 38
Table 2-4 Overall sensitivity, specificity, cost, and workload of alternate screening protocols at
important intervals (with discontinued screening after three negative results) ..................... 48
Table 2-5 Scenarios for patient compliance rates ......................................................................... 50
Table 2-6 Mathematical expressions of overall sensitivity, specificity, and cost for mild TBI
screening model ..................................................................................................................... 56
Table 3-1 Example of a rule-base for N = 2 screening test case ................................................... 68
Table 3-2 Rule-base for N = 4 screening test example ................................................................. 70
Table 3-3 Changes in diagnosis over time and according to defuzzification method .................. 71
Table 3-4 Illustrative portion of evaluation dataset ...................................................................... 72
Table 3-5 Category probabilities for MLR approach ................................................................... 77
xviii
Table 3-6 Changes in final diagnosis over time based on the method used ................................. 82
Table 3-7 Overall comparison of model performances ................................................................ 83
Table 4-1 Representation of basic discrete facility location models ............................................ 89
Table 4-2 Optimal allocation of patient zip codes to VISN 1 facilities for a 30-mile maximum
acceptable distance assuming estimated care demand in 2010 ........................................... 102
Table 4-3 Number of sleep beds, within-network and outside-network sleep apnea patients in
VISN 1 (January 2009 - May 2010) .................................................................................... 109
Table 4-4 Cost parameters used in mathematical models ........................................................... 119
Table 4-5 Multi-period model solution times based on different scenarios for case 1, 2, and 3 120
Table 4-6 Results for single-period model with 60-mile maximum acceptable travel distance . 122
Table 4-7 Optimal facility locations for single-period model based on estimated true demand 124
Table 4-8 Allocation of demand nodes to facilities for single-period model ............................. 125
Table 4-9 Results for multi-period model with 60-mile maximum acceptable travel distance .. 127
Table 4-10 Optimal facility locations for multi-period model.................................................... 129
Table 4-11 Allocation of demand nodes to facilities for multi-period model ............................ 130
xix
List of Abbreviations
ANN .............................. Artificial Neural Networks
CAPS .............................. Clinician-Administered PTSD scale
DoD .............................. Department of Defense
FL .............................. Fuzzy Logic
MLR .............................. Multinomial Logistic Regression
OEF/OIF .............................. Operations Enduring Freedom and Iraqi Freedom
PCL .............................. PTSD Clinician Checklist
PC-PTSD .............................. Primary Care PTSD Screen
PTSD .............................. Posttraumatic Stress Disorder
TBI .............................. Traumatic Brain Injury
VA .............................. Veterans Affairs
VHA .............................. Veterans Health Administration
VISN .............................. Veterans Integrated Service Network
Chapter 1 - Introduction
1
Introduction Chapter 1 -
This dissertation focuses on developing systems engineering models to optimize the overall
design, effectiveness, and capacity of large integrated healthcare systems for detecting and
treating two of most common military “silent” injuries, traumatic brain injury (TBI) and
posttraumatic stress disorder (PTSD), as well as a general health problem that is common among
veterans, sleep apnea. In this chapter, general information about the extended area of interest,
motivation behind this research, and outline of the dissertation are provided.
1.1 Silent Injuries
More than 2 million U.S. service members have been deployed since October 2001 for
Operations Enduring and Iraqi Freedom (OEF/OIF) in Afghanistan and Iraq, with many exposed
to prolonged periods of deployment-related stress and traumatic events (Tanielian and Jaycox,
2008; Congressional Budget Office, 2012). Fortunately, advances in protective equipment and
battlefield care have reduced the mortality rates that are historically lower than in earlier
prolonged conflicts, such as Vietnam and Korea (Regan, 2004; Warden, 2006). However, there
have been significant increases in a different kind of casualties – “silent” injuries such as mental
health conditions and cognitive impairments resulting from deployment experiences. Two of
Chapter 1 - Introduction
2
these conditions that affect military service members during deployment to a combat theater and
afterward are traumatic brain injury (TBI) and posttraumatic stress disorder (PTSD).
Both TBI and PTSD are referred as the signature wounds of Iraq and Afghanistan conflicts, with
disproportionately more casualties when compared to the physical injuries of combat. TBI is
caused by sudden trauma to the head and is commonly sustained by soldiers exposed to
explosions. PTSD, on the other hand, is an anxiety disorder induced by exposure to a traumatic
event, such as witnessing an injury or death. Unlike physical wounds of war, these conditions
often are invisible to the eye, remaining invisible to other service members, family members, and
society in general. These conditions affect mood, thoughts, and behavior and may go
unrecognized and unacknowledged indefinitely, resulting in serious medical and social
consequences (Tanielian and Jaycox, 2008; Congressional Budget Office, 2012).
1.1.1 Traumatic Brain Injury
A traumatic brain injury (TBI) is any non-penetrating or penetrating structural injury or
physiological disruption to the brain due to an external force (Langlois et al., 2006; Butler et al.,
2008). Falls (35.2%), motor vehicle and traffic accidents (17.3%), impacts (16.5%), and assaults
(10%) are the most common causes of civilian TBI. Among civilians, an estimated 1.7 million
people per year sustain a TBI, of which approximately 52,000 die, 275,000 are hospitalized, and
1.365 million, which is nearly 80%, are treated by an emergency department (Faul et al., 2010).
Military personnel and others working in combat zones also are at particular risk of TBI due to
encounters with explosions, rocket-propelled grenades, improvised explosive devices, and land
mines (Champion et al., 2009; Ling et al., 2009; Defense and Veterans Brain Injury Center,
2013). Since associated problems, such as related to thinking and memory, often are not visible
Chapter 1 - Introduction
3
and awareness among the general public is limited, TBI frequently is referred to as the “silent
epidemic” (Martin et al., 2008; McCrea, 2008; Vaishnavi et al., 2009; Faul et al., 2010).
TBI is identified as one of the signature injuries among U.S. troops serving in the Iraq and
Afghanistan wars, accounting for a larger proportion of casualties than in previous wars.
Improved protective equipment has reduced the incidence of penetrating head injuries and death,
but there has been a significant increase in the incidence of closed-head TBI. Some estimates of
military TBI are as high as 22%, with an estimated 50% of injured combat soldiers suffering
some form of TBI from Operations Enduring and Iraqi Freedom (OEF/OIF) (Okie, 2005;
Warden, 2006; Jackson et al., 2008; Tanielian and Jaycox, 2008; Wojcik et al., 2010; Smith et
al., 2013). As of May 2013, a total of 273,859 soldiers suffering from TBI were reported by the
Defense and Veterans Brain Injury Center (i.e., medical diagnoses of TBI among U.S. forces
since 2000), with 58% sustaining their injury while in the Army, 15% while in the Marines, 14%
while in the Navy, and 14% while in the Air Force (Defense and Veterans Brain Injury Center,
2013).
The severity of a TBI can be mild, moderate, or severe depending on the extent of the damage to
the brain. A person with a mild TBI may remain conscious or may experience a loss of
consciousness for a few seconds or minutes. Other symptoms of mild TBI include headache,
mental confusion, lightheadedness, dizziness, double and blurred vision or tired eyes, ringing in
the ears, bad or metallic taste in the mouth, fatigue or lethargy, a change in sleep patterns,
behavioral or mood changes, decreased coordination, and trouble with memory, concentration,
calculation, attention, or thinking. Worsening symptoms indicate a more severe injury, including
a more severe and permanent headache, repeated vomiting or nausea, personality change,
Chapter 1 - Introduction
4
convulsions or seizures, an inability to awaken from sleep, dilation of one or both pupils of the
eyes, slurred speech, weakness or numbness in the extremities, loss of coordination, and
increased confusion, restlessness, or agitation (National Institute of Neurological Disorders and
Stroke, 2013).
Most diagnosed TBIs (82%) are categorized as mild, while moderate (8%), severe (1%), or
penetrating wounds (2%) are less common (Defense and Veterans Brain Injury Center, 2013).
The diagnosis of TBI is relatively straightforward when symptoms are more visible, but mild
TBIs are difficult to detect, both by those afflicted and by healthcare professionals. Moreover, a
caregiver or a screening mechanism may attribute TBI symptoms to other medical or cognitive
conditions (Heegaard and Biros, 2007; Anderson-Barnes et al., 2010). In combat zones, TBI
often occurs in combination with more obviously life-threatening injuries; therefore, cases may
go unrecognized, potentially putting the individuals or their units at risk (Butler et al., 2008;
Tanielian and Jaycox, 2008). Additionally, when TBI occurs with no obvious symptoms, service
members might not seek medical treatment (Martin et al., 2008; McCrea, 2008). Many TBI
sufferers, especially if untreated, may endure medical, behavioral, and social consequences for
many years, even a lifetime. Any delays in treatment may compromise recovery or result in
significant cognitive, physical, and/or psychological impairment (Rao and Lyketsos, 2000; Maas
et al., 2008; Anderson-Barnes et al., 2010; Faul et al., 2010). Consequently, soldiers who are
exposed to blasts should be screened for TBI immediately following the event in order to
minimize medical complications.
The existence and severity of TBI are evaluated in several ways: subjective assessments, e.g.,
questions about dizziness; external signs, e.g., lacerations, fractures, fluids; physical experiences,
Chapter 1 - Introduction
5
e.g., duration of amnesia or unconsciousness; neurologic assessments, e.g., the Glasgow Coma
Scores (GCS), Military Acute Concussion Evaluation (MACE), the Automated
Neuropsychological Assessment Metrics (ANAM); and finally, clinical assessments, e.g.,
computed tomography (CT) scans, magnetic resonance imaging (MRI), and
electroencephalogram (EEG) (U.S. Department of Health and Human Services, 2006; Elder et
al., 2010; Maruta et al., 2010). In the military, testing typically follows a process of screening,
identification, evaluation, confirmation, and stratification by severity (Butler et al., 2008).
Proper treatment of mild TBI requires early identification, monitoring and management of
symptoms, and adequate rest. A person with signs of moderate or severe TBI, on the other hand,
should receive medical attention as soon as possible. Since little can be done to reverse the initial
brain damage caused by trauma, clinical care is first directed toward controlling secondary
effects of TBI, such as brain swelling, bleeding in the head, and poor cerebral blood flow, that
can lead to further neuronal damage. Primary concerns include insuring proper oxygen supply to
the brain and the rest of the body, maintaining adequate blood flow, and controlling blood
pressure. Approximately half of severely head-injured patients need surgery to remove or repair
ruptured blood vessels or bruised brain tissue. Following acute critical care, these patients
receive rehabilitation that involves individually tailored treatment programs in the areas of
physical therapy, occupational therapy, speech/language therapy, physical medicine,
psychology/psychiatry, and social support (Butler et al., 2008; National Institute of Neurological
Disorders and Stroke, 2013).
Disabilities resulting from a TBI depend upon the severity of the injury, the location of the
injury, and the age and general health of the patient. Some common disabilities include problems
Chapter 1 - Introduction
6
with cognition (thinking, memory, and reasoning), sensory processing (sight, hearing, touch,
taste, and smell), communication (expression and understanding), and behavior or mental health
(depression, anxiety, personality changes, aggression, acting out, and social inappropriateness).
More serious head injuries may result in stupor, an unconscious, unresponsive, unaware, and/or
unarousable state, and a vegetative or persistent vegetative state (National Institute of
Neurological Disorders and Stroke, 2013).
TBI incidence, screening processes, risk factors, treatment and care services, and consequences
have been studied extensively in the literature, including these by Nathanson et al. (2003),
Javouhey et al. (2006), Kaufman et al. (2006), Langlois et al. (2006), Côté et al. (2007), Guler et
al. (2008), Hoge et al. (2008), Jackson et al. (2008), Wu et al. (2008), Guler et al. (2009),
Elgmark Andersson et al. (2010), Faul et al. (2010), Rosenfeld and Ford (2010), Syam and Côté
(2010), Liao et al. (2012), and Asemota et al. (2013).
1.1.2 Posttraumatic Stress Disorder
Posttraumatic stress disorder (PTSD) is a severe and often disabling mental health disorder
resulting from traumatic events such as assaults, wars, disasters, terrorist attacks, and accidents
(American Psychiatric Association, 2000; Hoge et al., 2006; Milliken et al., 2007; Shalev, 2009).
PTSD is common among Americans with a lifetime prevalence of 3.7% in the general
population, yet only 7% of those affected seek treatment in their first year, with a lifetime
treatment-seeking rate of just over 50% (Kessler et al., 2005; Wang et al., 2005). In the military,
the prevalence is especially higher – up to 20% among Iraq and Afghanistan veterans – given
that exposure to combat increases the risk of PTSD (Prigerson et al., 2001; Hoge et al., 2004;
Ramchand et al., 2010).
Chapter 1 - Introduction
7
PTSD can occur in anyone who has experienced a traumatic event, including war veterans and
survivors of physical and sexual assaults, abuse, accidents, disasters, and many other serious
events. However, not everyone who has experienced a traumatic event eventually develop PTSD.
In fact, people who have not been through such traumas also can suffer from PTSD. Although it
is difficult to predict whether or not a person will develop PTSD, a number of factors that
increase the risk have been identified. These include having experienced dangerous events and
traumas, having a history of mental illness and/or a family history of mental illness, the severity
of the trauma experienced, and having little or no social support after the trauma (Kessler et al.,
1995; Brewin et al., 2000; U.S. Department of Health and Human Services, 2013). Additionally,
women are at increased risk of PTSD, with a lifetime prevalence of more than twice as high than
men, mostly because they are more likely to experience the kinds of trauma that can trigger the
condition such as sexual assaults (Halligan and Yehuda, 2000; Freedman et al., 2002; National
Comorbidity Survey-Replication, 2007; Shalev, 2009).
PTSD symptoms usually develop shortly after the traumatic event, but may not appear until
months or years have passed. In many survivors the symptoms subside, while they persist in
others in the form of chronic PTSD (Shalev, 2009). These symptoms can be grouped into three
categories, namely, re-experiencing, avoidance, and hyperarousal. Re-experiencing symptoms
include recurrent images, thoughts, flashbacks, or dreams about the traumatic event and may
cause problems in a person’s everyday routine. Avoidance is defined as making an effort to
avoid thoughts, feelings, conversations, places, or people that are reminders of the traumatic
event, feeling emotionally numb, losing interest in once-enjoyable activities, and having trouble
remembering the important details of the traumatic event. Finally, hyperarousal includes
symptoms of anxiety or heightened awareness of danger such as sleeplessness, irritability, being
Chapter 1 - Introduction
8
easily startled, or becoming overly vigilant (American Psychiatric Association, 2000; Bisson,
2007; U.S. Department of Health and Human Services, 2013).
PTSD is diagnosed based on signs and symptoms and a thorough psychological evaluation. Over
the past several decades, considerable progress has been made in the development and empirical
evaluation of assessment instruments, such as questionnaires, structured interviews, and
psychophysiological procedures, for measuring PTSD (Foa and Yadin, 2011). The most
commonly used PTSD assessment tools are Primary Care PTSD Screen (PC-PTSD), PTSD
Clinician Checklist (PCL), and Clinician Administered PTSD Scale (CAPS). PC-PTSD is a short
and simple screening questionnaire designed to detect the PTSD diagnosis in busy primary care
clinics. It includes four yes/no questions which focus on capturing PTSD symptom clusters. The
result of the PC-PTSD screening test is determined according to the number of “yes” answers
(i.e., cut-off score). To date, PC-PTSD and its diagnostic qualities have been exclusively studied,
indicating high sensitivity and specificity values for cut-off scores of 2 and 3 (Prins et al., 2003;
Seal et al., 2008; van Dam et al., 2010). PCL, on the other hand, is a 17-item self-report measure
of PTSD symptomatology, where respondents are asked to rate how much they have been
bothered by each symptom in the past month on a scale ranging from 1 (not at all) to 5
(extremely). Different scoring procedures are used to yield either a continuous measure of PTSD
symptom severity or a dichotomous (present/absent) indicator of diagnostic status. Dichotomous
scoring methods include either an overall cut-off score or a symptom cluster scoring approach
(American Psychiatric Association, 2000; Forbes et al., 2001; Norris and Hamblen, 2004; Keen
et al., 2008). A commonly used cut-off score is 50, which is suggested as the optimally efficient
cut-off score by Weathers et al. (1993). Finally, CAPS is widely considered the gold standard in
PTSD assessment and was designed to overcome the limitations of other available PTSD
Chapter 1 - Introduction
9
interviews. It is a 30-item structured interview, which requires clinicians to rate each PTSD
symptom’s frequency and intensity using a scale ranging from 0 to 4. The most frequently used
scoring rule is to record a symptom as present if it has a frequency of 1 or more and an intensity
of 2 or more (Blake et al., 1990; Blake et al., 1995; Weathers et al., 1999; Weathers et al., 2001;
Gray et al., 2004).
There are several treatments that were found to be effective with PTSD, including psychotherapy
(talk therapy), medications, or both. Psychotherapy involves talking with a mental health
professional either one-on-one or in a group. Talk therapy treatment for PTSD usually lasts 6 to
12 weeks, but can take more time. Many types of psychotherapy can help PTSD patients, some
of which target PTSD symptoms directly while others focus on social, family, or job-related
problems. Different therapies may be combined by therapists depending on each patient’s needs.
One of the helpful therapies is called cognitive behavioral therapy (CBT) which consists of
several parts, including exposure therapy, cognitive restructuring, and stress inoculation training.
Medications, on the other hand, help control PTSD symptoms such as sadness, worry, anger, and
feeling numb inside and also may make it easier to go through psychotherapy. The most
commonly prescribed medications for treating PTSD patients are sertraline and paroxetine, both
of which are antidepressants and also used to treat depression (U.S. Department of Health and
Human Services, 2013; VA National Center for PTSD, 2013).
Most of the problems associated with PTSD can be addressed by effective treatments (Foa et al.,
2008; U.S. Department of Veterans Affairs and Department of Defense, 2010). However, many
individuals with PTSD do not seek mental health care for various reasons – the stigma associated
with having a mental health problem, for example, or the inconvenience of undergoing additional
Chapter 1 - Introduction
10
evaluation and treatment (Congressional Budget Office, 2012). Instead, many of them use
primary care services, where PTSD may be underdetected or undertreated (Schnurr et al., 2013).
If remain untreated or undertreated, PTSD results in significant morbidity and long-term
disability (Schnurr et al., 2006; Hermann et al., 2012). Moreover, people with PTSD are at very
high risk to develop a number of other mental health disorders (Orsillo et al., 1996; Keane and
Kaloupek, 1997; Shalev et al., 1998), the most common ones being anxiety disorders (73.3%),
depression (68.6%), and substance use disorders (10.5%) (Magruder et al., 2005). The co-
occurrence of PTSD and another disorder may have a negative impact on the treatment of PTSD
and compromise recovery.
There are numerous studies in the literature that focus on PTSD prevalence (Kessler et al., 1995;
Hoge et al., 2004; Magruder et al., 2005; Ramchand et al., 2010; Shiner et al., 2012), risk factors
(Brewin et al., 2000; Halligan and Yehuda, 2000; Freedman et al., 2002), symptoms (Bisson,
2007), consequences (Murdoch et al., 2005; Smith et al., 2005; Schnurr and Lunney, 2011),
screening processes (Forbes et al., 2001; Prins et al., 2003; Gray et al., 2004), and treatment
services (Rosenheck and Fontana, 2007; Hermes et al., 2012; Shiner et al., 2012; Schnurr et al.,
2013).
1.2 Sleep Apnea
Sleep apnea is a type of sleep disorder characterized by pauses in breathing (called “apneas”) or
instances of shallow or infrequent breathing (called “hypopneas”) during sleep. Each pause in
breathing may last from at least ten seconds to minutes and occur 5 to 30 times or more an hour
(Ancoli-Isreal and Avalon, 2006; Erman, 2006; U.S. Department of Health and Human Services,
Chapter 1 - Introduction
11
2012). There are two types of sleep apnea, namely, obstructive and central sleep apnea. In
obstructive sleep apnea, breathing is abnormal because of narrowing or closure of the throat,
while in central sleep apnea, it is because of a change in the breathing control and rhythm
(Schmidt-Nowara, 2012). Obstructive sleep apnea is more common among adults with the
estimates of prevalence being in the range of 3% to 7% in the general population (Morgenthaler
et al., 2006; Punjabi, 2008). Military veterans are at particularly high risk of sleep disorders due
to hazardous working conditions, inconsistent work hours, harsh environments, routine exposure
to loud noises, and crowded sleeping spaces, with the most frequent diagnosis being obstructive
sleep apnea (Seelig et al., 2010; Mysliwiec et al., 2013). Also, recent studies suggest that the
increased incidence of sleep disorders among military personnel is potentially related to PTSD,
depression, anxiety, or mild TBI (Sharafkhaneh et al., 2005; Lewis et al., 2009; Amin et al.,
2010; McLay et al., 2010).
Sleep apnea is a serious condition that affects a person’s ability to safely perform normal daily
activities as well as his/her long term health and has been linked with high blood pressure, heart
problems, metabolic syndrome, diabetes, stroke, fatigue, headaches, snoring, daytime
drowsiness, and memory problems (Ancoli-Isreal and Avalon, 2006; Erman, 2006). Although
anyone can develop sleep apnea, there are certain factors that increase the risk, including
increasing age, male sex, obesity, family history, menopause, craniofacial abnormalities, and
certain health behaviors such as cigarette smoking or alcohol use (Young et al., 2004; Punjabi,
2008; Schmidt-Nowara, 2012; U.S. Department of Health and Human Services, 2012).
The main symptoms of sleep apnea are loud snoring, fatigue, and daytime sleepiness. However,
some people have no symptoms or may not be aware of the symptoms such as snoring. Fatigue
Chapter 1 - Introduction
12
and sleepiness, on the other hand, often are attributed to other reasons such as overwork and
increasing age. As a result, some people may not recognize that they have a problem and
therefore do not seek medical care. Other symptoms may include restless sleep, awakening with
choking, gasping, or smothering, morning headaches, dry mouth or sore throat, awakening
unrested, memory impairment, difficulty concentrating, and low energy, which again may go
unrecognized or be attributed to other reasons (Schmidt-Nowara, 2012).
The diagnosis of sleep apnea is based on the conjoint evaluation of clinical symptoms (e.g.,
excessive daytime sleepiness and fatigue) and of the results of a formal sleep study such as
polysomnography or reduced channels home based test. The overnight polysomnogram, the
standard diagnostic test for obstructive sleep apnea, involves simultaneous recordings of multiple
physiologic signals during sleep, including the electroencephalogram, electrooculogram,
electromyogram, oronasal airflow, and oxyhemoglobin saturation. These recordings allow
identification and classification of sleep-related apneas and hypopneas. Sleep apnea severity is
typically assessed with the apnea–hypopnea index (AHI), which is the number of apneas and
hypopneas per hour of sleep. Several additional measures of disease severity that characterize the
degree of nocturnal hypoxemia (e.g., average oxyhemoglobin desaturation) and extent of sleep
fragmentation (i.e., arousal frequency) also are used in the clinical and research areas (American
Academy of Sleep Medicine, 1999; Punjabi, 2008; Schmidt-Nowara, 2012). The polysomnogram
is attended in a laboratory by a sleep technician. However, home portable monitoring devices
also are available to diagnose sleep apnea in individuals with a high probability of at least
moderate or severe sleep apnea who do not have other illnesses or sleep disorders that may
interfere with the results (Lee et al., 2008; Schmidt-Nowara, 2012).
Chapter 1 - Introduction
13
The goals of treating sleep apnea are to restore regular breathing during sleep and to relieve
symptoms such as loud snoring and daytime sleepiness. For milder cases, home remedies and
lifestyle changes such as losing weight, quitting smoking, adjusting sleep position, avoiding
alcohol and other sedatives, and using nasal sprays or allergy medicines to keep nasal passages
open at night may be enough to treat or reduce symptoms. If these changes do not improve the
symptoms or sleep apnea is moderate to severe, other treatments such as breathing devices or
surgery are recommended. The most effective treatment for moderate to severe sleep apnea is to
use a mechanical device – continuous positive airway pressure (CPAP) – to keep the upper
airway open during sleep. A CPAP device uses an air-tight attachment to the nose, typically a
mask, connected to a tube and a blower which generates the pressure (Gay et al., 2006; Schmidt-
Nowara, 2012; U.S. Department of Health and Human Services, 2012). Although CPAP is the
most common and reliable method of treating sleep apnea, some people find it uncomfortable
and experience problems while using it. In these cases, other devices such as bi-level positive
airway pressure (BPAP), expiratory positive airway pressure (EPAP), or oral devices may be
recommended. Surgery, on the other hand, is usually only an option for patients who cannot
tolerate or do not improve with non-surgical treatments such as CPAP or oral devices (Mayo
Clinic, 2012; American Sleep Apnea Association, 2012a).
Although there are relatively simple, inexpensive, and effective modalities for diagnosing and
treating sleep apnea, the vast majority of patients remain undiagnosed and therefore untreated
because of the lack of awareness by the public and healthcare professionals. If left untreated,
sleep apnea can have serious and life-shortening consequences, including high blood pressure,
heart disease, stroke, diabetes, depression, memory problems, weight gain, impotence, and
headaches. Moreover, untreated sleep apnea may be responsible for job impairment and motor
Chapter 1 - Introduction
14
vehicle accidents, as studies have shown that people with severe sleep apnea are more than twice
as likely to be involved in a motor vehicle accident as people without this condition (Ellen et al.,
2006; George, 2007; Lee et al., 2008; Schmidt-Nowara, 2012; American Sleep Apnea
Association, 2012b).
There are many studies in the literature that focus on sleep apnea prevalence, risk factors,
symptoms, consequences, diagnosis, and treatment, including these by Sin et al. (1999), Beninati
and Sanders (2001), Sharafkhaneh et al. (2004), El-Ad and Lavie (2005), Kjelsberg et al. (2005),
Ellen et al. (2006), Ferguson et al. (2006), Gay et al. (2006), Freire et al. (2007), George (2007),
Lee et al. (2008), Punjabi (2008), Matthews and Aloia (2009), Yaggi and Strohl (2010), Lloberes
et al. (2011), Ryan et al. (2011), Mannarino et al. (2012), Vozoris (2012), Madani et al. (2013),
and Mysliwiec et al. (2013).
1.3 Motivation for Work
Although “silent” injuries have generated widespread concern among policymakers, recent
studies suggest that fundamental gaps remain about the mental health and cognitive needs of
U.S. service members returning from Iraq and Afghanistan and the sufficiency of the care
systems available to meet these needs. There is a strong belief that the Veterans Health
Administration (VHA), the healthcare system within the Department of Veterans Affairs (VA),
and the healthcare system for active-duty personnel within the Department of Defense (DoD)
cannot adequately screen, diagnose, and treat the service members and veterans affected by TBI
and PTSD, and yet little of the existing research is useful in guiding policy with regard to how
Chapter 1 - Introduction
15
best to improve access to and the delivery of mental health care to those in need (Hoge et al.,
2004; Tanielian and Jaycox, 2008; Congressional Budget Office, 2012).
The main part of this dissertation focuses on illustrating the use of systems engineering
approaches to improve the effectiveness and cost of screening and diagnostic services for TBI
and PTSD as well as patients’ access to these services within large integrated healthcare systems,
such as VHA. This research is of great importance to the VHA – and potentially to the DoD as
well – as it focuses on ensuring U.S. servicemen receive proper care for a large and growing
healthcare concern. Substantial improvements in other healthcare sectors have resulted from
systems engineering, suggesting significant opportunities for mental health care process
improvements and cost reductions.
As a large integrated healthcare system, VHA provides comprehensive medical services to the
veterans with not only combat-related conditions but also a broad range of other general health
problems. Given that most veterans are exposed to situations and circumstances that have long-
lasting effects on their health and thus more likely to develop chronic conditions (Yu et al.,
2003), there are significant opportunities for further improvements both in quality and cost of
care provided with the use of systems engineering methods to optimize the care processes for
long-term health/general care problems along with those for mental health conditions. These
opportunities also are illustrated in this dissertation, using sleep apnea as a case study.
The remainder of this dissertation is organized as follows. Chapter 2 presents several probability
and Monte Carlo simulation models to determine the overall accuracy and cost of current and
proposed screening processes for PTSD within the VHA, along with the illustration of a mild
Chapter 1 - Introduction
16
TBI sequential screening process. In Chapter 3, three types of predictive diagnostic models –
fuzzy logic, logistic regression, and neural networks – are developed to properly categorize TBI
patients into the most likely severity state and a numerical example is provided for illustration
purposes. Chapter 4 presents several single and multi-period integer programming models to
determine the optimal locations and capacities for a range of specialty care services within the
VHA, namely, PTSD treatment and sleep apnea testing services, which minimize overall system
cost and total travel distances while providing appropriate within-network access. Finally, a
summary of conclusions, limitations, and recommendations for future work are addressed in
Chapter 5.
Chapter 2 - Sequential Screening Models
17
Sequential Screening Models Chapter 2 -
Among U.S. military members, mental disorders account for significant morbidity, disability,
healthcare utilization, and attrition from military service. Veterans with untreated mental health
problems may face severe consequences to their overall health and ability to fully reintegrate into
the community, exacerbating the potential for chronic mental health conditions. Posttraumatic
stress disorder (PTSD), for instance, causes significant physical and social problems along with
tremendous medical and social costs if untreated or undertreated. This chapter presents several
probability and Monte Carlo simulation models to develop a better understanding of current
PTSD screening processes within the Veterans Health Administration (VHA), to determine the
overall system accuracy, cost, and the total amount of resources for the current and proposed
configurations, and to conduct a scenario analysis in order to thoroughly explore the potential
improvements in the early diagnosis and care of veterans with PTSD. Also illustrated is a
sequential screening process for traumatic brain injury (TBI) in order to give an insight into how
the general approach is applied to other disorders.
2.1 Background
Mental health problems are among the most important contributors to the burden of disease and
disability worldwide (Brundtland, 2000). Mental disorders are one of the leading causes of
Chapter 2 - Sequential Screening Models
18
disability in the U.S., with as many as 30% of the adult population suffering from a diagnosable
mental disorder in a given year (Kessler et al., 1994; Kessler et al., 2005). The excess disability
that is attributed to mental disorders is a result of high prevalence and chronicity, early age of
onset, and the fact that they result in serious impairments (WHO International Consortium in
Psychiatric Epidemiology, 2000). The magnitude of this burden also is attributed to the fact that
fewer than half of individuals with these disorders receive treatment or initial treatment is
generally delayed for many years, sometimes decades (Kessler et al., 1999; Kessler et al., 2001;
Kohn et al., 2004; Wang et al., 2005). Undiagnosed and untreated mental illnesses have serious
medical and social consequences, including but not limited to an illness that is more severe and
difficult to treat, the development of co-occurring mental illnesses, worsening disability,
unemployment, substance abuse, homelessness, and suicide (Norquist and Regier, 1996; Kessler
et al., 2001). Individuals with untreated mental illnesses face an increased risk of having chronic
medical conditions, yet they are less likely to follow-up with treatment for their health problems
and ultimately die earlier than others without these conditions (Ciechanowski et al., 2000; Gehi
et al., 2005; Colton and Manderscheid, 2006; Manderscheid et al., 2008). Moreover, the
consequences extend beyond the personal cost and into the societal cost, as people with untreated
mental illnesses earn less money and are prone to social isolation (Crisp et al., 2000; Goetzel et
al., 2002). If identified early and treated effectively, most people with serious mental illnesses
can significantly reduce the impact of their illness, improve their earning potential, and integrate
into their community (Katon et al., 2005; Schennach et al., 2012; Schnurr and Lunney, 2012).
The effective treatment of mental health problems in primary care patients has been plagued for
decades by the under-diagnosis and the under-treatment of mental and substance related
disorders. Although there are marked gains in the identification and treatment of some mental
Chapter 2 - Sequential Screening Models
19
health disorders in many primary care settings, progress is slower for the anxiety disorders,
particularly posttraumatic stress disorder (PTSD) (Magruder et al., 2005). A well-designed and
effective screening program can help properly identify individuals with PTSD and give them
access to effective treatments in a timely fashion. Mathematical modeling approaches, such as
probability and simulation models, can be very useful at helping policy makers, clinicians, and
researchers establish a better understanding of current PTSD screening processes, develop more
effective alternatives both in terms of quality and cost, and further perform a scenario analysis to
explore the possible improvements. These types of mathematical models have been extensively
discussed by Benneyan et al. in the case of cervical cancer (Benneyan and Kaminsky, 1996;
Kaminsky et al., 1997). Other applications include different health issues such as breast cancer
(Ozekici and Pliska, 1991; Baker, 1998; Lee and Zelen, 1998; Warwick and Duffy, 2005;
Maillart et al., 2008; Ayer et al., 2012), colorectal cancer (Knudsen et al., 2012), bladder cancer
(Kent et al., 1989), hypertension (Donner et al., 1979), Down’s syndrome (Malone et al., 2005),
and depression and dementia (Thibault and Steiner, 2004).
2.2 PTSD Screening Process within the VHA
PTSD is a serious mental health disorder which debilitates physical and social well-being and
produces tremendous medical and social costs if untreated or undertreated (Schnurr et al., 2006;
Hermann et al., 2012). The estimated prevalence of lifetime PTSD is increasingly higher in the
military when compared to the general population, as it is one of the signature injuries in the Iraq
and Afghanistan wars (Kessler et al., 2005; Magruder et al., 2005; Tanielian and Jaycox, 2008).
In the Veterans Affairs (VA) population, PTSD is consistently associated with poorer health,
decreased earnings, and social isolation (Schnurr and Green, 2004; Murdoch et al., 2005).
Chapter 2 - Sequential Screening Models
20
Fortunately, effective treatments, many of which were tested in the VHA, can address these
problems if the veterans with PTSD are identified and treated in a timely fashion (Smith et al.,
2005; Davis et al., 2012; Schnurr and Lunney, 2012). Therefore, the VA has invested significant
resources in the development of a comprehensive PTSD screening system and instituted a yearly
screening program for all primary care patients beginning in 2010 (Shiner et al., 2012). Since
then, over 98% of primary care patients received screening in compliance with the protocol (U.S.
Department of Veterans Affairs, 2012). Currently, each veteran is supposed to be screened
annually by the Primary Care PTSD screen (PC-PTSD) for the first five years after his/her most
recent date of separation from active duty, and then every five years afterwards. However, the
date of separation is not known for all VA users and thus PC-PTSD screening continues annually
beyond the first five years for those with an unknown date of separation. A clinical exam, which
has a sensitivity (i.e., probability of diagnosing a PTSD patient) of 0.465 and a specificity (i.e.,
probability of testing negative in the absence of PTSD) of 0.966 (Magruder et al., 2005),
confirms or disconfirms a positive result on the PC-PTSD. A schematic presentation of the
current PTSD screening process in the VA is given in Figure 2-1.
Chapter 2 - Sequential Screening Models
21
PC-PTSD1
Yes (with prob. rs)Year 1
Year 2
Year j
No(with prob.
1-rs)
Negative
Positive
...
- Yearly for the first 5 years after the date of separation- Then, every five years if date of separation is known,
yearly otherwise
Patient does not take PC-PTSD in the corresponding year
Patient takesPC-PTSD?
PC-PTSD2
Yes (with prob. rs)
No(with prob.
1-rs)
Negative
Positive
Negative
Patient takesPC-PTSD?
PC-PTSDj
Yes (with prob. rs)
No(with prob.
1-rs)
Negative
Positive
Patient takesPC-PTSD?
Negative
Positive
Final determination of “Positive”
Final determination of “Negative”
...
...
Unstructured clinical exam
(with PPV of 0.64 & with NPV of 0.93)
Patient comes inwith symptoms?
Figure 2-1 Current PTSD screening process in the VA
While the PC-PTSD is a useful screening measure and applied appropriately in the VA, it
represents only a first step in properly identifying PTSD patients. Confirmatory tests, such as the
self-administered PTSD Checklist (PCL) and Clinician-Administered PTSD scale (CAPS) also
play a role in PTSD identification after initial screening with the PC-PTSD. Therefore, this study
proposes a more systematic approach where PCL and CAPS are used as confirmatory tests, as
shown in Figure 2-2. The main characteristics of PC-PTSD, PCL, and CAPS are summarized in
Table 2-1, with more detailed information presented earlier in Section 1.1.2.
Chapter 2 - Sequential Screening Models
22
PC-PTSD1
Yes (with prob. rs)Year 1
Year 2
Year j
No(with prob.
1-rs)
Negative
Positive
...
Patient takesPC-PTSD?
PC-PTSD2
Yes (with prob. rs)
No(with prob.
1-rs)
Negative
Positive
Negative
Patient takesPC-PTSD?
PC-PTSDj
Yes (with prob. rs)
No(with prob.
1-rs)
Negative
Positive
Patient takesPC-PTSD?
...
...
- Yearly for the first 5 years after the date of separation (DOS)
- Then, every 5 years if DOS is known, yearly otherwise
Patient does not take PC-PTSD in the corresponding year
Unspecified (with prob. u)
Patient takesPCL?
No (with prob. 1-rp)
PCL
Patient takesCAPS?
Final determination of “Positive”
Yes(with prob. rp)
CAPS
Yes(with prob. rc)
Final determination of “Negative”
Negative
Negative
No (with prob. 1-rc)
Positive
Positive
Figure 2-2 Proposed PTSD screening process in the VA
Table 2-1 Main characteristics of the PC-PTSD, PCL, and CAPS screening tests
Test Cut-off score Sensitivity* Specificity*
Patient compliance
rate
Avg cost
Time (minutes) By whom
Avg Stdev
PC-PTSD 2 0.91 0.72
0.90 $1 0.5 0.5 Administrative staff 3 0.84 0.90
PCL 50 0.82 0.83
0.30 $25 8 8 Self-administered (interpreted by a
clinician) DSM 1.00 0.92 Both 0.60 0.99
CAPS (Gold standard) DSM 1.00 1.00 0.05 $200 70 50 Trained clinician
*Shiner et al. (2012)
Chapter 2 - Sequential Screening Models
23
2.3 Methodology
2.3.1 Notation and Assumptions
A mathematical model was developed to determine the overall system sensitivity, specificity,
and total screening and false diagnosis costs under current and proposed process configurations
for PTSD screening. The following notation is used in the probabilistic model, with regards to
Test characteristics:
αs = probability that a PC-PTSD will determine a truly positive patient as negative,
αp = probability that a PCL will determine a truly positive patient as negative,
αc = probability that a CAPS will determine a truly positive patient as negative,
βs = probability that a PC-PTSD will determine a truly negative patient as positive,
βp = probability that a PCL will determine a truly negative patient as positive,
βc = probability that a CAPS will determine a truly negative patient as positive,
u = probability that a PCL cannot make a definite diagnosis,
rs = patient compliance rate for PC-PTSD,
rp = patient compliance rate for PCL, and
rc = patient compliance rate for CAPS;
Cost parameters:
ks = average cost of a PC-PTSD,
kp = average cost of a PCL,
kc = average cost of a CAPS,
kFP = average cost of a false-positive final result, and
kFN = average cost of a false-negative final result per year;
Chapter 2 - Sequential Screening Models
24
Policy performance measures:
j = analysis horizon,
pj = overall sensitivity after j years,
qj = overall specificity after j years,
pFN|Pos, j = probability of not detecting a truly positive patient after j years,
pFP|Neg, j= probability of identifying a truly negative patient as positive after j years,
pTP, j = probability that any patient yields a true-positive after j years,
pFN, j = probability that any patient yields a false-negative after j years,
pTN, j = probability that any patient yields a true-negative after j years,
pFP, j = probability that any patient yields a false-positive after j years,
Is = number of PC-PTSD tests per any given patient,
Ip = number of PCL tests per any given patient,
Ic = number of CAPS tests per any given patient,
pcon+ = probability that diagnosis is confirmed by either PCL or CAPS given that the
patient is positive,
pcon− = probability that diagnosis is confirmed by either PCL or CAPS given that the
patient is negative,
ps_nc+ = probability that patient takes PC-PTSD but result remains unconfirmed by
neither PCL nor CAPS given that he/she is positive,
ps_nc− = probability that patient takes PC-PTSD but result remains unconfirmed by
neither PCL nor CAPS given that he/she is negative,
pp_nc+ = probability that patient takes PCL but result remains unconfirmed given that
he/she is positive, and
pp_nc− = probability that patient takes PCL but result remains unconfirmed given that
he/she is negative.
The overall system sensitivity is defined as the probability that a truly positive patient will be
determined correctly to be positive within the analysis horizon. Similarly, the overall system
specificity is defined as the probability that a truly negative patient will be classified correctly as
Chapter 2 - Sequential Screening Models
25
negative at the end of the analysis horizon. In the proposed configuration, each patient is
supposed to be screened by the PC-PTSD during j ≥1 years. After a negative determination by
the PC-PTSD in year i < j, no further action is taken and the patient is scheduled for another
screen the next year. In any year i < j, if the PC-PTSD determines the patient to be positive,
he/she is sent to the PCL for confirmation. If PCL cannot make a definite diagnosis (with
probability u), the patient is sent to the CAPS.
The cost estimates for ks, kp, kc, kFP, and kFN are based on discussions with several psychiatrists in
the VA. Specifically, the estimate for the cost of each test (ks, kp, and kc) is based on the
personnel needed for administration. False-positive and false-negative costs (kFP and kFN), on the
other hand, are more difficult to estimate and consist of treatment and disability costs as well as
cost of anxiety and inconvenience caused to the patient. These costs vary from patient to patient
and significantly different results might be obtained using different estimates. In order to make
more accurate estimates, several cases for both false-positive and false-negative results were
identified and for each case, minimum, average, and maximum cost values were determined, as
given in Table 2-2. The probability distributions of these costs also are illustrated in Figure 2-3.
Chapter 2 - Sequential Screening Models
26
Table 2-2 Minimum, average, and maximum cost estimates for a false-positive and false-negative result under the assumption of different cases
Case Definition Percentage Cost
Min Avg Max
Fals
e-
posit
ive
1 Initial evaluation 40% - $300 - 2 Minimum treatment 20% $1,000 $3,000 $5,000 3 Ongoing treatment 20% $6,000 $15,000 $30,000
4 Partial disability and extensive treatment 10% $30,000 $40,000 $55,000
5 Full disability and extensive treatment 10% - $60,000 -
Fals
e-ne
gativ
e 1 Minimal symptoms 25% - $1,000 - 2 Partial disability 50% - $15,000 -
3 Full disability (dysfunctional) 25% - $30,000 -
Figure 2-3 Probability distribution of cost of (a) a false-positive and (b) a false-negative result
0.000000
0.000025
0.000050
0.000075
0.000100
0.000125
0.000150
0.000175
0.000200
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
$0 $5 $10 $15 $20 $25 $30 $35 $40 $45 $50 $55 $60 $65
Prob
abili
ty (c
ases
2 &
3 &
4)
Prob
abili
ty (c
ases
1 &
5)
False-positive cost (in thousands)
(a) Probability distribution of cost of a false-positive result
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
0.50
$0 $3 $6 $9 $12 $15 $18 $21 $24 $27 $30 $33
Prob
abili
ty
False-negative cost (in thousands)
(b) Probability distribution of cost of a false-negative result
Evaluated & detected negative
Got minimum treatment
Ongoing treatment
Became partially dysfunctional & got extensive treatment
Became dysfunctional & got extensive treatments and disability payments
Survived with minimal
symptoms
Became partially dysfunctional
Became dysfunctional
kFP = $13,720
kFN = $15,250
Chapter 2 - Sequential Screening Models
27
Other model assumptions include
(1) Each PC-PTSD, PCL, and CAPS test has a patient compliance rate (i.e., not all patients
take these tests even though it is required) and these rates are assumed to remain constant
over time.
(2) Each PC-PTSD, PCL, and CAPS test has a sensitivity of 1-αs, 1-αp, and 1-αc,
respectively. Thus, the probability that a PC-PTSD, PCL, and CAPS test will correctly
determine a truly positive patient to be positive is 1-αs, 1-αp, and 1-αc, respectively.
These sensitivities are assumed to remain constant over time.
(3) Each PC-PTSD, PCL, and CAPS test has a specificity of 1-βs, 1-βp, and 1-βc,
respectively. Thus, the probability that a PC-PTSD, PCL, and CAPS test will correctly
determine a truly negative patient to be negative is 1-βs, 1-βp, and 1-βc, respectively.
These specificities are assumed to remain constant over time.
(4) All tests are assumed to be independent of each other.
(5) The expected number of PC-PTSD tests per positive patient and per negative patient are
denoted, respectively, as E[Is | Positive] and E[Is | Negative]. Similarly, the expected
number of PCL and CAPS tests per positive patient and per negative patient are denoted,
respectively, as E[Ip | Positive], E[Ic | Positive], E[Ip | Negative], and E[Ic | Negative].
(6) Screening and false diagnosis costs are assumed to remain constant over time.
Chapter 2 - Sequential Screening Models
28
2.3.2 Overall System Sensitivity
2.3.2.1 Probabilities of Detecting Truly Positive Patients and of a False-Negative
The overall sensitivity is a combined function of the individual PC-PTSD, PCL, and CAPS
sensitivities, the patient compliance rates, the length of the analysis horizon, and the probability
that a PCL cannot make a definite diagnosis. Note that a truly positive patient can be given a
final correct positive determination in several manners. For example, assuming that the patient
complies with the overall screening process, if the PC-PTSD in any year correctly determines the
patient to be positive, the PCL subsequently confirms this determination. Alternatively, after a
positive determination from the PC-PTSD, if the PCL cannot make a definite diagnosis, the
CAPS confirms the positive determination.
By considering all possible probabilistic manners in which a truly positive patient is correctly
identified as positive, a mathematical model of the overall probability of detecting a truly
positive patient was developed based on the assumptions and notation given in Section 2.3.1.
Thus, the probability of detecting a truly positive patient in j years is shown in Appendix A to be
Overall sensitivity = Probability of detecting a truly positive patient
pj = P{Positive determination|Patient is positive}
=�1−�1−rs(1−αs)rp�1−u(1−rc)��
j��1−αp−u�1−αp−rc(1−αc)��
1−u(1−rc) . (2-1)
The probability of a false-negative (FN), traditionally defined as the conditional probability of
incorrectly identifying a truly positive patient as negative in j years, simply is
Chapter 2 - Sequential Screening Models
29
Probability of a FN = Probability of not detecting a truly positive patient
pFN|Pos, j = P{Negative determination|Patient is positive}
= 1 − Overall sensitivity = 1 − pj
=�1−rs(1−αs)rp�1−u(1−rc)��
j�1−αp−u�1−αp−rc(1−αc)��+(1−u)αp+urcαc
1−u(1−rc) . (2-2)
The unconditional probabilities of a true-positive (TP) and a false-negative (FN) result after j
years screening of any patient whose true state is not known a priori are the above rates
multiplied by the prevalence, p, of PTSD in the population under consideration:
P{Any patient yields a TP} = pTP, j = p × pj
= p�1−�1−rs(1−αs)rp�1−u(1−rc)��
j��1−αp−u�1−αp−rc(1−αc)��
1−u(1−rc) , (2-3)
P{Any patient yields a FN} = 𝑝FN, j = p × pFN|Pos, j
= p�1−rs(1−αs)rp�1−u(1−rc)��
j�1−αp−u�1−αp−rc(1−αc)��+(1−u)αp+urcαc
1−u(1−rc) . (2-4)
As an example, given a prevalence of p = 0.115 (Magruder et al., 2005), PC-PTSD sensitivity of
1-αs = 0.84 (with a cut-off score of 3), PCL sensitivity of 1-αp = 1 (with DSM criteria), CAPS
sensitivity of 1-αc = 1 (with DSM criteria), compliance rate of rs = 0.90, rp = 0.30, and rc = 0.05
for PC-PTSD, PCL, and CAPS, respectively, and assuming that PCL cannot make a definite
diagnosis with a probability of u = 0.10, then the probabilities of incorrectly determining a truly
positive patient to be negative and of incorrectly determining any patient to be negative in the
first year (j = 1) are
Chapter 2 - Sequential Screening Models
30
P{Negative determination|Patient is positive} = pFN|Pos, 1
=�1−rs(1−αs)rp�1−u(1−rc)��
j�1−αp−u�1−αp−rc(1−αc)��+(1−u)αp+urcαc
1−u(1−rc)
=�1−(0.90)(0.84)(0.30)�1−(0.10)(1−0.05)��
1�1−(0.10)�1−(0.05)(1)��+(1−0.10)(0)+(0.10)(0.05)(0)
1−(0.10)(1−0.05)
= 1 − �1 − �(0.90)(0.16) + 1 − 0.90�1� 0.30 �1 − 0.10�1 − 0.05(1)��
= 0.7947
and
P{Any patient yields a FN} = 𝑝FN, 1 = p × pFN|Pos, 1 = (0.115)(0.7947) = 0.0914.
2.3.2.2 Expected Number and Distribution of False-Negatives
If np positive patients are screened independently, then for any given set of conditions the
number of correct positive identifications is modeled with a binomial distribution with an
expected value and standard deviation of nppj and �nppj �1 − pj�, respectively. Similarly, the
number of undetected positive patients out of n patients of unknown medical status also has a
binomial distribution, here with an expected value and standard deviation of npFN, j and
�npFN, j �1 − pFN, j�, respectively. (For completeness, also note that the companions to these two
random variables – the number of false-negative diagnoses per np positive patients and the
number of correct positive diagnoses per n patients of unknown status – are binomial with the
same variances as above and with expectations nppFN | P, j and npTP, j, respectively.)
Chapter 2 - Sequential Screening Models
31
These expectations, variances, and distributions can be used to predict the relative frequencies of
the number of events of interest which will occur over time, such as the number of false-
negatives per n patients per year. For example, Figure 2-4 shows the probability distribution of
the number of false-negative diagnoses per n = 10,000 patients for the previous example. Note
that the expected number of false-negative diagnoses in the first year is
Expected number of FN diagnoses per 10,000 patients =
npFN, 1 = 10,000 × 0.0914 = 913.9579,
and the standard deviation is
�npFN, 1 �1 − pFN, 1� = �10,000 × 0.0914 × (1 − 0.0914) = √830.43 ≈ 28.8171,
which agrees with the location and spread of the corresponding probability mass in Figure 2-4.
Figure 2-4 Probability distributions of number of false diagnoses per 10,000 patients in the first year (p = 0.115, j = 1, u = 0.10, αs = 0.16, αp = 0, αc = 0, βs = 0.10, βp = 0.08, βc = 0, rs = 0.90, rp = 0.30, rc = 0.05)
0.00
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.10
0 75 150 225 300 375 450 525 600 675 750 825 900 975 1050
Prob
abili
ty (f
requ
ency
)
Number of false diagnoses per 10,000 patients
False-negativesFalse-positives
Chapter 2 - Sequential Screening Models
32
2.3.3 Overall System Specificity
2.3.3.1 Probabilities of Detecting Truly Negative Patients and of a False-Positive
Similar to the above, a mathematical model of the overall probability of detecting a truly
negative patient was developed based on the assumptions and notation given earlier in Section
2.3.1. Thus, the probability of identifying a truly negative patient as negative in j years is shown
in Appendix B to be
Overall specificity = Probability of correctly classifying a truly negative patient
qj = P{Negative determination|Patient is negative}
=�1−rsβsrp�1−u(1−rc)��
j�(1−u)βp+urcβc�+1−βp−u�1−βp−rc�1−βc��
1−u(1−rc) . (2-5)
Also as above, the probability of a false-positive (FP), traditionally defined as the conditional
probability of incorrectly identifying a truly negative patient as positive, is
Probability of a FP = P{Positive determination|Patient is negative}
pFP|Neg, j = 1 − qj =�1−�1−rsβsrp�1−u(1−rc)��
j��(1−u)βp+urcβc�
1−u(1−rc) , (2-6)
and the corresponding unconditional true-negative (TN) and false-positive (FP) probabilities are
P{Any patient yields a TN} = pTN, j = (1 − p) × qj
= (1 − 𝑝)��1−rsβsrp�1−u(1−rc)��
j�(1−u)βp+urcβc�+1−βp−u�1−βp−rc�1−βc��
1−u(1−rc) � (2-7)
Chapter 2 - Sequential Screening Models
33
and
P{Any patient yields a FP} = pFP, j = (1− p) × pFP|Neg, j
= (1 − p)��1−�1−rsβsrp�1−u(1−rc)��
j��(1−u)βp+urcβc�
1−u(1−rc) �. (2-8)
As an example, given a prevalence of p = 0.115, PC-PTSD specificity of 1-βs = 0.90 (with a cut-
off score of 3), PCL specificity of 1-βp = 0.92 (with DSM criteria), CAPS specificity of 1-βc = 1
(with DSM criteria), compliance rate of rs = 0.90, rp = 0.30, and rc = 0.05 for PC-PTSD, PCL,
and CAPS, respectively, and assuming that PCL cannot make a definite diagnosis with a
probability of u = 0.10, then the probabilities of incorrectly determining a truly negative patient
to be positive and of incorrectly determining any patient to be positive in the first year (j = 1) are
P{Positive determination|Patient is negative} = pFP|Neg, 1
=�1−�1−rsβsrp�1−u(1−rc)��
j��(1−u)βp+urcβc�
1−u(1−rc)
=�1−�1−(0.90)(0.10)(0.30)�1−(0.10)(1−0.05)��
1��(1−0.10)(0.08)+(0.10)(0.05)(0)�
1−(0.10)(1−0.05)
= 1 − ��(0.90)(0.90) + 1 − 0.90�1
+ �1 − �(0.90)(0.90) + 1 − 0.90�1�
× 0.30 �0.92 − 0.10�0.92 − (0.05)(1)���
= 0.0019
and
P{Any patient yields a FP} = pFP, 1 = (1 − p) × pFP|Neg, 1 = (1− 0.115)(0.0019) = 0.0017.
Chapter 2 - Sequential Screening Models
34
2.3.3.2 Expected Number and Distribution of False-Positives
Again, if nn negative patients are screened independently, then the number of correct negative
identifications is modeled with a binomial distribution with parameters nn and qj, and the
expected number and standard deviation of correct identifications per nn truly negative patients,
nnqj and �nnqj �1 − qj�, respectively, can be examined. Similarly, the number of undetected
negative patients out of n patients of unknown medical status also has a binomial distribution,
here with an expected value and standard deviation of npFP, j and �npFP, j �1 − pFP, j�,
respectively. (Again, note that the companions to these two random variables – the number of
false-positive diagnoses per nn negative patients and the number of correct negative diagnoses
per n patients of unknown status – are binomial with the same variances as above and with
expectations nnpFP | N, j and npTN, j, respectively.)
Figure 2-4 also shows the probability distribution of the number of false-positive diagnoses again
per n = 10,000 patients for the previous example. Note that the expected number of false-positive
diagnoses in this situation is
Expected number of FP diagnoses per 10,000 patients =
npFP, 1 = 10,000 × 0.0017 = 17.2044,
and the standard deviation is
�npFP, 1 �1 − pFP, 1� = �10,000 × 0.0597 × (1 − 0.0597) = √17.1748 ≈ 4.1442,
Chapter 2 - Sequential Screening Models
35
which again agrees with the corresponding probability mass in Figure 2-4.
2.3.4 Expected Cost
Based on the previous assumptions and notation, the expected cost per n patients screened after j
years is shown in Appendix C to be
ECj = Expected cost of all PC-PTSD tests per n patients + Expected cost of all PCL tests per n patients + Expected cost of all CAPS tests per n patients + Expected cost of all false-positives per n patients + Expected cost of all false-negatives per n patients
= �ks × Expected number of PC-PTSD tests per n patients� + �kp × Expected number of PCL tests per n patients� + (kc × Expected number of CAPS tests per n patients) + �kFP × Expected number of false-positives per n patients� + �kFN × Expected number of false-negatives per n patients�
= ksE�Is, n� + kpE�Ip, n� + kcE�Ic, n� + kFPE[FP] + kFNjE[FN]
= ksn�p �∑ i �ps_nc+ �
i−1pcon+ ∑ �(a+i−1)!
a!(i−1)!� (1 − rs)aj−i
a=0ji=1 + 𝑗ps_nc
+ �1− pcon+ �
j−1�
+(1− p) �∑ i �ps_nc− �
i−1pcon− ∑ �(a+i−1)!
a!(i−1)!� (1 − rs)aj−i
a=0ji=1 + 𝑗ps_nc
− �1 − pcon− �
j−1��
+kpn�p �∑ i �pp_nc+ �
i−1pcon+ ∑ �(a+i−1)!
a!(i−1)!� �1 − rs(1 − αs)rp�
aj−ia=0
ji=1 + 𝑗pp_nc
+ �1− pcon+ �
j−1�
+(1− p) �∑ i �pp_nc− �
i−1pcon− ∑ �(a+i−1)!
a!(i−1)!� �1 − rsβsrp�
aj−ia=0
ji=1 + 𝑗pp_nc
− �1− pcon− �
j−1��
+kcn � urc1−u(1−rc)� �p �1 − �1 − pcon
+ �j�+ (1 − p) �1 − �1 − pcon
− �j��
+kFPn(1 − p)��1−�1−rsβsrp�1−u(1−rc)��
j��(1−u)βp+urcβc�
1−u(1−rc) �
+kFNjnp��1−rs(1−αs)rp�1−u(1−rc)��
j�1−αp−u�1−αp−rc(1−αc)��+�(1−u)αp+urcαc�
1−u(1−rc) �. (2-9)
Chapter 2 - Sequential Screening Models
36
As an example, the expected cost per n = 10,000 screened patients for the previous situation
(with p = 0.115, j = 1, 1-αs = 0.84, 1-αp = 1, 1-αc = 1, 1-βs = 0.90, 1-βp = 0.92, 1-βc = 1, rs =
0.90, rp = 0.30, rc = 0.05, and u = 0.10) and with cost estimates ks = $1, kp = $25, kc = $200, kFP =
$13,720, and kFN = $15,250 is (note that pcon+ , pcon
− , ps_nc+ , ps_nc
− , pp_nc+ , and pp_nc
− were calculated as
given in Appendix C)
EC1 = ksn �p �pcon+ + ps_nc
+ �+ (1 − p) �pcon− + ps_nc
− ��
+kpn �p �pcon+ + pp_nc
+ �+ (1 − p) �pcon− + pp_nc
− ��
+kcn � urc1−u(1−rc)� �p�pcon
+ � + (1 − p)�pcon− ��
+kFPn(1 − p)��rsβsrp�1−u(1−rc)���(1−u)βp+urcβc�
1−u(1−rc) �
+kFNjnp��1−rs(1−αs)rp�1−u(1−rc)���1−αp−u�1−αp−rc(1−αc)��+�(1−u)αp+urcαc�
1−u(1−rc) �
= ks(10,000)�(0.115)(0.2053 + 0.6947) + (1 − 0.115)(0.0244 + 0.8756)� +kp(10,000)�(0.115)(0.2053 + 0.0215) + (1 − 0.115)(0.0244 + 0.0026)� +kc(10,000) � (0.10)(0.05)
1−(0.10)(1−0.05)� �(0.115)(0.2053) + (1− 0.115)(0.0244)� +kFP(10,000)(1 − 0.115)(0.90)(0.10)(0.30)(1− 0.10)(0.08) +kFN(10,000)(0.115) �1 − (0.90)(0.84)(0.30)�1− (0.10)(1 − 0.05)��
= ($1)(10,000)(0.90) + ($25)(10,000)(0.0499) + ($200)(10,000)(0.00025) +($13,720)(10,000)(0.0017) + ($15,250)(10,000)(0.0914)
= $9,000 + $12,494.25 + $499.77 + $236,044.37 + $13,937,857.98 = $14,195,896.36.
Although it is not pursued in this study, the variance of total cost per n screened patients also can
be obtained analytically, and the distribution has been examined via computer simulation. Figure
2-5 illustrates a distribution of 5,000 simulation replications of the cost per 10,000 screened
patients for the above situation, which fits the normal distribution best at a significance level of
Chapter 2 - Sequential Screening Models
37
0.05 (with a p-value of 0.388). Further simulation results are summarized later in Table 2-3,
indicating close agreement with the analytic results derived in this section.
Figure 2-5 Probability distribution of total cost per 10,000 patients (obtained via simulation) (p = 0.115, j = 1, u = 0.10, ks = $1, kp = $25, kc = $200, kFP = $13,720, kFN = $15,250, αs = 0.16, αp = 0, αc = 0,
βs = 0.10, βp = 0.08, βc = 0, rs = 0.90, rp = 0.30, rc = 0.05, 5000 replications)
0.000
0.005
0.010
0.015
0.020
0.025
0.030
0.035
0.040
0.045Pr
obab
ility
(fre
quen
cy)
Total cost per 10,000 patients (in millions)
Chapter 2 - Sequential Screening Models
38
Table 2-3 Summary of analytic distributions of cost, workload, and accuracy with comparison to simulation results – 5,000 replications (p = 0.115, u = 0.10, ks = $1, kp = $25, kc = $200, kFP = $13,720, kFN = $15,250, αs = 0.16, αp =
0, αc = 0, βs = 0.10, βp = 0.08, βc = 0, rs = 0.90, rp = 0.30, rc = 0.05)
Year (j)
Per 10,000 patients (n = 10,000)
Analytic results Simulation program results
Expected value Stdev Expected
value Stdev 95% CI on E[X]
j = 1
Total cost $14,195,896 $443,124.47 $14,182,851 $540,481.09 ± $14,981.37
Workload (# screenings): by admins (PC-PTSD)
by clinicians (PCL) by trained clinicians (CAPS)
9,000.00 499.7700
2.4989
30.0000 21.7897 1.5806
9,000.07 499.9034
2.4748
30.3069 22.0563 1.5754
± 0.8401 ± 0.6114 ± 0.0437
Accuracy: # false-negatives # false-positives
913.9579 17.2044
28.8171 4.1442
913.5562 17.1826
28.4409 4.1328
± 0.7883 ± 0.1146
j = 3
Total cost $34,569,800 $1,194,883 $34,551,996 $1,382,077 ± $38,309.23
Workload (# screenings): by admins (PC-PTSD)
by clinicians (PCL) by trained clinicians (CAPS)
25,827.17 1332.3215
6.6616
63.3129 34.4606 2.5801
25,826.99 1331.4600
6.6116
62.5802 34.8994 2.5811
± 1.7346 ± 0.9674 ± 0.0715
Accuracy: # false-negatives # false-positives
577.2752 50.3623
23.3227 7.0787
577.4552 50.3798
23.3051 7.0805
± 0.6460 ± 0.1963
j = 5
Total cost $47,592,761 $1,800,087 $47,580,935 $1,998,281 ± $55,389.51
Workload (# screenings): by admins (PC-PTSD)
by clinicians (PCL) by trained clinicians (CAPS)
41,369.48 2005.5960 10.0280
111.1222 41.0852 3.1651
41,371.51 2004.5810
9.9778
109.7363 41.3518 3.2076
± 3.0417 ± 1.1462 ± 0.0889
Accuracy: # false-negatives # false-positives
364.6193 81.9196
18.7437 9.0138
364.9010 81.9864
18.7438 9.0851
± 0.5196 ± 0.2518
2.4 Scenario Analysis
2.4.1 Effect of Scoring Methods on Overall System Accuracy, Cost, and Workload
The expected value expressions above also can be used to more precisely examine the joint
effect of various scoring methods (i.e., test performances) and the length of screening horizon on
Chapter 2 - Sequential Screening Models
39
the overall system sensitivity, specificity, cost, and clinician workload. For this purpose, four
different screening protocols (i.e., alternate scenarios for the test performances) were developed
and their overall performance in terms of accuracy and cost was compared to the current process
as well as the baseline which is the performance of clinician identification of PTSD before yearly
PC-PTSD was implemented, given a prevalence of p = 0.115, compliance rate of rs = 0.90, rp =
0.30, and rc = 0.05 for PC-PTSD, PCL, and CAPS, respectively, and assuming that PCL cannot
make a definite diagnosis with a probability of u = 0.10. In the first protocol, the scoring method
with the highest specificity is assumed to be used for PC-PTSD while the one with the highest
sensitivity is chosen for PCL. In other words, the first protocol aims to eliminate the truly
negative patients in the first step of the screening process (i.e., PC-PTSD) by using the scoring
method with the highest specificity, and then to successfully confirm the truly positive patients in
the second step (i.e., PCL or CAPS) by using the scoring method with the highest sensitivity.
The second protocol is the opposite of the first one while in the third protocol, the scoring
method with the highest sensitivity is used both for PC-PTSD and PCL. Finally, the last protocol
illustrates the annual screening with PCL instead of PC-PTSD. These screening protocols are
illustrated in Figure 2-6 with their overall sensitivity, specificity, and screening cost per patient
after 5 years.
Chapter 2 - Sequential Screening Models
40
Figure 2-6 Comparison of past, current, and proposed screening protocols for PTSD identification in the VA
Figure 2-7 illustrates the effect of the test performances on the overall system sensitivity and
specificity over a 20-year screening horizon, showing that the addition of the PC-PTSD
increased the sensitivity of PTSD patient identification in the VA by almost 15% while
maintaining a specificity of over 97%, when compared to annual clinical exams alone (i.e.,
baseline). Including triggered testing with the PCL and CAPS in the case of positive results adds
further improvements in sensitivity as yearly screening continues (i.e., after 3 years for protocols
1, 3, and 4, after 10 years for protocol 2). However, these proposed screening protocols might
result in losses of specificity if repeated over a 20-year horizon.
Chapter 2 - Sequential Screening Models
41
Figure 2-7 Effect of scoring method on overall (a) sensitivity and (b) specificity over a 20-year screening horizon
Figure 2-8 plots the expected number of false-negatives and false-positives per 10,000 patients
against years. As expected, protocol 4, which proposes yearly PCL screening, performs best in
terms of sensitivity and specificity (i.e., generates less false-negative and false-positive results).
Protocol 3 has the second best sensitivity but the worst specificity, while protocol 2 has the
second best specificity but the worst sensitivity. Protocol 1, on the other hand, gives relatively
good results in terms of both the number of false-negatives and false-positives.
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Prob
abili
ty o
f det
ectin
g tr
ue-p
ositi
ves
Year (j)
(a) Overall system sensitivity
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Prob
abili
ty o
f det
ectin
g tr
ue-n
egat
ives
Year (j)
(b) Overall system specificity
Baseline Current Protocol 1 Protocol 2 Protocol 3 Protocol 4
Chapter 2 - Sequential Screening Models
42
Figure 2-8 Effect of scoring method on expected number of (a) false-negatives and (b) false-positives over a 20-year screening horizon
Figure 2-9 compares the expected screening and total costs per 10,000 patients over the years,
indicating that protocol 4, which achieves the best sensitivity and specificity, also has the lowest
total cost (i.e., total of screening, false-negative, and false-positive costs). Its screening cost,
however, is significantly higher (up to $300,000 more per year), as well as the number of CAPS
tests needed (up to 500 more CAPS per year) when compared to the other proposed protocols;
this might generate an excessive workload considering that there may not be enough clinicians
0
100
200
300
400
500
600
700
800
900
1,000
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Num
ber o
f fal
se-n
egat
ives
pe
r 10,
000
patie
nts
Year (j)
(a) Expected number of false-negatives
0
100
200
300
400
500
600
700
800
900
1,000
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Num
ber o
f fal
se-p
ositi
ves
per 1
0,00
0 pa
tient
s
Year (j)
(b) Expected number of false-positives
Baseline Current Protocol 1 Protocol 2 Protocol 3 Protocol 4
Chapter 2 - Sequential Screening Models
43
with a working knowledge of PTSD in the VA to perform that many CAPS tests. Protocol 1, on
the other hand, achieves a 29.94% and 56.65% decrease in the number of false-negatives and
false-positives in the first 5 years when compared to the current process, and generates a
reasonable screening cost and clinician workload, as illustrated in Figure 2-10.
Figure 2-9 Effect of scoring method on expected (a) screening cost and (b) total cost over a 20-year screening horizon
$0
$6
$12
$18
$24
$30
$36
$42
$48
$54
$60
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Scre
enin
g co
st p
er 1
0,00
0 pa
tient
s (in
mill
ions
)
Year (j)
(a) Expected screening cost
$0
$35
$70
$105
$140
$175
$210
$245
$280
$315
$350
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Tota
l cos
t per
10,
000
patie
nts
(in m
illio
ns)
Year (j)
(b) Expected total cost
Baseline Current Protocol 1 Protocol 2 Protocol 3 Protocol 4
Chapter 2 - Sequential Screening Models
44
Figure 2-10 Decomposition of expected (a) total cost and (b) amount of screening tests assuming protocol 1
The above examples illustrate the inherent tradeoff that decision makers face – that yearly
screening significantly increases the probability of detecting true-positives, but has the opposite
effect on the probability of identifying true-negatives. Figure 2-7, however, shows that the
magnitude of this effect on reducing specificity is lower than on increasing sensitivity. On the
other hand, Figure 2-8 illustrates that, when compared to the current process, protocol 1
decreases the number of false-negatives significantly, but generates more false-positives as
yearly screening continues. Therefore, to consider “stopping yearly screening after a number of
consecutive negative results” controls the increase in the number of false-positives while
$0
$8
$16
$24
$32
$40
$48
$56
$64
$72
$80
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Cost
per
10,
000
patie
nts (
in m
illio
ns)
Year (j)
(a) Cost decomposition
Total costScreening costFN costFP cost
0
13
26
39
52
65
78
91
104
117
130
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Num
ber o
f scr
eeni
ng te
sts p
er 1
0,00
0 pa
tient
s (in
thou
sand
s)
Year (j)
(b) Screening volume
Total number of screening testsNumber of PC-PTSD testsNumber of PCL testsNumber of CAPS tests
Chapter 2 - Sequential Screening Models
45
maintaining the improvement achieved in terms of false-negatives. Figure 2-11 illustrates the
tradeoff between the stopping criteria (i.e., number of negatives required to stop screening) and
the expected number of false diagnoses and cost for protocol 1, indicating that discontinued
screening after three or four negative results achieves a significant decrease in the number of
false-positives and the screening cost, with a slight increase in the number of false-negatives and
total cost. Note that stopping screening earlier decreases the number of false-positives further,
but causes an enormous increase in the number of false-negatives and ultimately, the total cost.
Figure 2-11 Tradeoff between stopping criteria and expected (a) number of false diagnoses, and (b) cost at the end of screening horizon (j = 20) assuming protocol 1
0
50
100
150
200
250
300
350
400
450
500
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Num
ber o
f fal
se d
iagn
oses
pe
r 10,
000
patie
nts
Stopping criteria (number of negative results required to stop screening)
(a) Expected number of false diagnoses
False-negatives
False-positives
$0
$25
$50
$75
$100
$125
$150
$175
$200
$225
$250
$0
$17
$34
$51
$68
$85
$102
$119
$136
$153
$170
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Scre
enin
g co
st p
er 1
0,00
0 pa
tient
s (in
thou
sand
s)
Tota
l cos
t per
10,
000
patie
nts
(in m
illio
ns)
Stopping criteria (number of negative results required to stop screening)
(b) Expected cost
Total cost
Screening cost
Chapter 2 - Sequential Screening Models
46
Figure 2-12 and 2-13 illustrate that under the assumption of protocol 1, a significant decrease in
the number of false-positives is achieved by stopping yearly screening after three negative
results, resulting in a reasonable increase in the number of false-negatives and the total cost.
Furthermore, overall sensitivity, specificity, cost, and clinician workload of all screening
protocols with discontinued screening after three negative results are summarized later in Table
2-4.
Figure 2-12 Effect of discontinued screening after three negative results on expected (a) number of false-negatives and (b) number of false-positives assuming protocol 1
0
100
200
300
400
500
600
700
800
900
1,000
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Num
ber o
f fal
se n
egat
ives
pe
r 10,
000
patie
nts
Year (j)
(a) Expected number of false-negatives
Protocol 1 - stop after three negatives
0
Protocol 1
Current
0
100
200
300
400
500
600
700
800
900
1,000
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Num
ber o
f fal
se p
ositi
ves
per 1
0,00
0 pa
tient
s
Year (j)
(b) Expected number of false-positives
Protocol 1 - stop after three negatives
Current
Protocol 1
Chapter 2 - Sequential Screening Models
47
Figure 2-13 Effect of discontinued screening after three negative results on expected (a) screening cost and (b) total cost assuming protocol 1
$0
$2
$4
$6
$8
$10
$12
$14
$16
$18
$20
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Scre
enin
g co
st p
er 1
0,00
0 pa
tient
s (in
mill
ions
)
Year (j)
(a) Expected screening cost
Protocol 1 Protocol 1 - stop after three negatives
Current
$0
$21
$42
$63
$84
$105
$126
$147
$168
$189
$210
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Tota
l cos
t per
10,
000
patie
nts
(in m
illio
ns)
Year (j)
(b) Expected total cost
Protocol 1 - stop after three negatives
Protocol 1
Current
Chapter 2 - Sequential Screening Models
48
Table 2-4 Overall sensitivity, specificity, cost, and workload of alternate screening protocols at important intervals (with discontinued screening after three negative results)
Performance measure* Year 1 Year 3 Year 5 Year 10 Year 20
Cur
rent
stat
e
Sensitivity 0.3796 0.5260 0.5470 0.5249 0.5252 Specificity 0.9892 0.9787 0.9786 0.9802 0.9802
Expected cost: Screening
Total
$1,131,700
$13,321,237
$3,202,506
$33,049,644
$5,289,318
$53,381,554
$9,898,422
$103,562,981
$19,118,444 $207,489,577
Workload (# screenings): by admins (PC-PTSD)
by clinicians (clinical exam)
6,749.38 3,749.83
19,642.72 10,609.54
32,579.59 17,522.46
61,100.88 32,791.07
118,143.39 63,334.34
Prot
ocol
1
Sensitivity 0.2054 0.4981 0.6803 0.8545 0.9126 Specificity 0.9981 0.9943 0.9933 0.9932 0.9932
Expected cost: Screening
Total
$21,994
$14,183,610
$60,469
$34,542,493
$74,790
$47,285,676
$82,013
$64,961,683
$84,246
$83,421,454 Workload (# screenings):
by admins (PC-PTSD) by clinicians (PCL)
by trained clinicians (CAPS)
8,999.98 499.57
2.52
25,828.22 1,331.66
6.75
30,893.70 1,687.45
8.55
32,105.33 1,918.78
9.69
32,410.12 1,993.00
10.05
Prot
ocol
2
Sensitivity 0.1340 0.3188 0.4306 0.5379 0.5782 Specificity 0.9993 0.9981 0.9976 0.9974 0.9974
Expected cost: Screening
Total
$33,732
$15,301,406
$90,947
$40,813,516
$118,357
$61,731,975
$129,712
$105,332,802
$132,520
$181,515,773 Workload (# screenings):
by admins (PC-PTSD) by clinicians (PCL)
by trained clinicians (CAPS)
9,000.64 951.19
4.76
24,764.01 2,545.40
12.74
32,037.39 3,319.82
16.62
34,327.71 3,668.55
18.35
34,729.19 3,761.20
18.81
Prot
ocol
3
Sensitivity 0.2225 0.5298 0.7153 0.8931 0.9602 Specificity 0.9946 0.9848 0.9804 0.9794 0.9793
Expected cost: Screening
Total
$33,733
$14,334,987
$90,956
$34,437,863
$118,348
$46,407,750
$129,718
$60,972,252
$132,533
$71,671,584 Workload (# screenings):
by admins (PC-PTSD) by clinicians (PCL)
by trained clinicians (CAPS)
8,999.60 951.01
4.79
24,763.53 2,545.63
12.76
32,034.03 3,319.43
16.64
34,325.83 3,668.59
18.39
34,727.65 3,761.40
18.85
Prot
ocol
4
Sensitivity 0.2700 0.6111 0.7927 0.9407 0.9863 Specificity 1.0000 1.0000 1.0000 1.0000 1.0000
Expected cost: Screening
Total
$325,244
$13,135,053
$895,878
$29,882,365
$1,067,509
$38,675,902
$1,123,298
$47,948,290
$1,138,391
$52,893,837 Workload (# screenings):
by clinicians (PCL) by trained clinicians (CAPS)
8,998.83 501.37
25,724.37 1,263.84
30,194.43 1,563.24
31,014.51 1,739.68
31,197.53 1,792.26
*Per 10,000 patients
Chapter 2 - Sequential Screening Models
49
2.4.2 Effect of False Diagnosis Costs on the Total Cost of Screening Protocols
The total cost includes false-negative and false-positive costs, which are subject to considerable
uncertainty and subjective discussion. Even though a range of possible cost estimates, given in
Table 2-2, are considered to be as realistic as possible, this uncertainty still is a limitation and the
exploration of how the overall system is affected by the changes in these costs is especially
important. To this end, Figure 2-14 illustrates the effect of the ratio between false-negative and
false-positive costs on the total cost under the assumption of protocol 1 with various stopping
criteria, indicating that the false-negative cost has more influence on the total cost than the false-
positive cost. The total cost significantly increases as the false-negative cost increases, but the
same increase in the false-positive cost does not make any significant change.
Figure 2-14 Effect of false-negative/false-positive cost ratio on total cost assuming protocol 1 with various stopping criteria
2.4.3 Effect of Compliance Rates on Overall System Accuracy and Cost
The patient compliance rates in the above examples (Table 2-1) were determined based on
discussions with several psychiatrists in the VA and intended to be as realistic as possible.
$0
$10
$20
$30
$40
$50
$60
$70
$80
$90
10:1 5:1 2:1 1:1 1:2 1:5 1:10
Tota
l cos
t per
10,
000
patie
nts
(in m
illio
ns)
False-negative/false-positive cost ratio (kFN/kFP)
Stop after 3 negativesStop after 5 negativesStop after 10 negativesNo stopping
FN cost held constant as FP cost increases
FP cost held constant as FN cost increases
Chapter 2 - Sequential Screening Models
50
However, it is important to emphasize that these rates are not known with certainty and that it is
necessary to explore how the system accuracy and cost are affected by various compliance
values. For this purpose, several compliance rate scenarios, given in Table 2-5, were developed
based on currently available data. The pessimistic and optimistic scenarios represent the lower
and upper bounds, respectively, while realistic scenario reflects the current situation (i.e., the one
used in the previous examples). Also shown for comparison is the “perfect” situation which
assumes that all patients fully comply with each test (i.e., patients take the tests when it is
required).
Table 2-5 Scenarios for patient compliance rates
Scenario Compliance rates
PC-PTSD (rs) PCL (rp) CAPS (rc) Pessimistic (intolerable) 0.75 0.30 0.01
Realistic (current) 0.90 0.30 0.05 Optimistic (ideal) 1.00 0.50 0.25
Perfect (unrealistic) 1.00 1.00 1.00
Figure 2-15 and 2-16 illustrate the effect of patient compliance on the overall system sensitivity,
specificity, and screening and total cost under the assumption of protocol 1 with discontinued
screening after three negative results, indicating that, as expected, the number of false-negatives
and thus total cost decrease as each test’s compliance rate increases. The number of false-
positives and the cost of screening, however, increase significantly as patients comply more with
each test, primarily because of conducting more PCL tests (note that only 10% of patients who
take PCL cannot be determined as positive or negative and are tested with CAPS which is
considered the gold standard for PTSD diagnosis). More specifically, improving patient
compliance results in up to 99.18% and 91.48% decrease in the number of false-negatives and
Chapter 2 - Sequential Screening Models
51
the total cost, while generating an increase up to 268.62% and 324.53% in the number of false-
positives and the cost of screening, respectively. Of note, even though these increases seem quite
high, the number of false-positives and the cost of screening per year are still 7.41% and 97.33%
less on average, respectively, when compared to the current process, even assuming the “perfect”
patient compliance which generates the biggest increases.
Figure 2-15 Effect of compliance rates on expected (a) number of false-negatives and (b) number of false-positives assuming protocol 1 with discontinued screening after three negative results
0
100
200
300
400
500
600
700
800
900
1,000
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Num
ber o
f fal
se-n
egat
ives
per
10
,000
pat
ient
s
Year (j)
(a) Expected number of false-negatives
0
20
40
60
80
100
120
140
160
180
200
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Num
ber o
f fal
se-p
ositi
ves p
er
10,0
00 p
atie
nts
Year (j)
(b) Expected number of false-positives
Pessimistic compliance Realistic compliance Optimistic compliance Perfect compliance
Chapter 2 - Sequential Screening Models
52
Figure 2-16 Effect of compliance rates on expected (a) screening cost and (b) total cost assuming protocol 1 with discontinued screening after three negative results
2.4.4 Effect of PTSD Prevalence on the Predictive Value of Screening Protocols
The utility of a screening test and the rationale behind the screening programs are strongly
determined by the prevalence of the disease in the population (Harris et al., 2001). Therefore, the
performance of the screening protocols for various PTSD prevalence rates also is analyzed. For
instance, Figure 2-17 illustrates the changes in the positive and negative predictive values of
protocol 1 with discontinued screening after three negatives for a range of PTSD population
$0
$20
$40
$60
$80
$100
$120
$140
$160
$180
$200
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Scre
enin
g co
st p
er 1
0,00
0 pa
tient
s (in
thou
sand
s)
Year (j)
(a) Expected screening cost
$0
$10
$20
$30
$40
$50
$60
$70
$80
$90
$100
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Tota
l cos
t per
10,
000
patie
nts
(in m
illio
ns)
Year (j)
(b) Expected total cost
Pessimistic compliance Realistic compliance Optimistic compliance Perfect compliance
Chapter 2 - Sequential Screening Models
53
prevalence (i.e., from low to very high). As expected, using the same protocol in a population
with higher prevalence increases the positive predictive value significantly while resulting in a
slightly decreased negative predictive value. In other words, even though the corresponding
protocol has good performance characteristics (i.e., sensitivity of 0.91 and specificity of 0.99), its
positive predictive value is fairly low when applied to a population with lower prevalence,
suggesting that the VA is an ideal place to evaluate the PTSD screening protocols given the
higher prevalence when compared to the general population (Kessler et al., 2005; Magruder et
al., 2005; Seal et al., 2007; Shiner et al., 2012).
Figure 2-17 Effect of PTSD prevalence on predictive value of protocol 1 with discontinued screening after three negative results
2.5 Other Possible Applications: A Sequential Screening Model for Mild TBI
While the main focus of this chapter is PTSD, similar sequential screening models can be
developed for a range of other mental health conditions. In order to give an insight into how the
general approach is applied to other disorders, this section illustrates a sequential screening
probability model developed for mild traumatic brain injury (TBI). As one of the common
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
1.10
0.00 0.05 0.10 0.15 0.20 0.25
Pred
ictiv
e va
lue
PTSD prevalence (p)
Positive predictive
value
Negative predictive value
Low Medium High Very high
Chapter 2 - Sequential Screening Models
54
military injuries which occurs with no visible symptoms and may go unrecognized and untreated,
mild TBI would be a good example of how sequential screening can help reduce the number of
unrecognized cases and improve the effectiveness and cost.
Similar to the above PTSD screening processes, the proposed mild TBI screening process
consists of a number of screening tests which are conducted at predetermined time intervals (may
vary depending on the extent of the injury) and a confirmation test which is used to follow-up on
a positive determination of a screening test. As usual, the confirmation test tends to be more
thorough and expensive than the screening tests and aims to confirm any positive diagnosis made
by the screening tests. Confirmation of a negative diagnosis also might be necessary, especially
in some settings such as combat zones given that the complicated nature and strict restrictions of
war conditions. To this end, a decision stage is included in the screening process, which decides
whether or not a patient should take the confirmation test even though he/she is determined as
negative by all screening tests. Given that the soldiers work as teams (troops) and members of
the same team are subject to the same or similar external effects that may cause TBI (i.e., mines,
grenade blasts, etc.), the existence of a soldier with TBI in a small team is assumed to increase
the likelihood of finding TBI on his teammates. Therefore, the screening process is extended
such that the patients who are determined as negative by all screening tests are rescreened by the
confirmation test if at least one of their teammates has diagnosed with TBI recently. The
schematic presentation of the proposed screening process is given in Figure 2-18.
Chapter 2 - Sequential Screening Models
55
Screening test 1
...
Is there someonewith TBI in thepatient’s team?
Final determination of “Negative”
Final determination of “Positive”
Screening test 2
Negative
Negative
Negative
Screening test j
Negative
Confirmation test
Positive
Positive
Positive
No Negative Positive
Yes (with prob. r1 or r2)
Figure 2-18 A sequential screening process for mild TBI
Mathematical expressions for overall accuracy and cost of mild TBI screening process can be
built using joint, conditional, and expected value probability laws, similar to those for the PTSD
screening process. Again, by considering all possible probabilistic manners in which a truly
positive [negative] patient is correctly identified as positive [negative] and the expected number
of screenings and false diagnoses, a mathematical model of the overall sensitivity, specificity,
and expected total cost was developed under the assumption of both non-homogeneous and
homogeneous screening tests, as summarized in Table 2-6. Derivation of these mathematical
expressions is given in Appendix D along with the model assumptions and notation.
Chapter 2 - Sequential Screening Models
56
Table 2-6 Mathematical expressions of overall sensitivity, specificity, and cost for mild TBI screening model
Overall sensitivity Overall specificity
Expected total cost
Non
-hom
ogen
eous
sc
reen
ing
test
s
pj = �1 −∏ αiji=1 (1 − r1)� (1 − αc) qj = 1 − βc �1 −∏ �1 − βi�
ji=1 (1 − r2)�
ECj = ksn �p �(1 − α1) + ∑ �i∏ αki−1k=1 (1 − αi)�
j−1i=2 + j∏ αk
j−1k=1 �
+(1 − p)�β1 + ∑ �i∏ �1 − βk�i−1k=1 βi�
j−1i=2 + j∏ �1 − βk�
j−1k=1 ��
+kcn �1 − �p∏ αiji=1 (1 − r1) + (1 − p)∏ �1 − βi�
ji=1 (1 − r2)��
+kFPn(1 − p)βc �1 −∏ �1 − βi�ji=1 (1 − r2)�
+kFNnp �αc + (1 − αc)∏ αiji=1 (1 − r1)�
Hom
ogen
eous
sc
reen
ing
test
s
pj = �1 − αsj(1 − r1)�(1 − αc) qj = 1 − βc �1 − �1 − βs�
j(1 − r2)�
ECj = ksn�p �1−αsj
1−αs� + (1 − p)�1−�1−βs�
j
βs��
+kcn�1 − �pαsj(1 − r1) + (1 − p)�1 − βs�
j(1 − r2)��
+kFPn(1 − p)βc �1 − �1 − βs�j(1 − r2)�
+kFNnp�αc + (1 − αc)αsj(1 − r1)�
where (1–αi) and (1–αc) are the sensitivities, and (1–βi) and (1–βc) are the specificities for each screening test i and the confirmation test, p is the population incidence rate, n is the number of patients to be screened, r1 and r2 are the rescreening rates, and ks, kc, kFP, and kFN is the cost of a screening test, the confirmation test, a false-positive result, and a false-negative result.
Similar to the case of PTSD screening, the above expressions can be used to more precisely
examine the joint effect of various test accuracies and number of screening tests on the overall
system sensitivity, specificity, and total cost. Figure 2-19, for example, illustrates the competing
effect of additional screenings on overall sensitivity and specificity, and the rate at which they
approach their respective bounds (confirmation test sensitivity and specificity which need not be
the same), given an incidence of p = 0.15 and rescreening rates of r1 = 0.40 and r2 = 0.10 and
assuming that, for simplicity, all screening tests have identical sensitivities and specificities.
Note that the confirmation test sensitivity is an upper bound on the overall system sensitivity as
the number of screening tests and/or their sensitivities increase. Similarly, the overall system
specificity is bounded below by the confirmation test specificity. Thus, no screening policy of
Chapter 2 - Sequential Screening Models
57
this type, even with many near-perfect [very poor] screening tests, can have higher [lower]
system sensitivity [specificity] than the final confirmation test (Benneyan and Kaminsky, 1996).
As these examples illustrate, significant improvements in the probability of detecting truly
positive patients clearly are possible in many cases by adding more screening tests. It, however,
has the opposite effect of decreasing the probability of correctly classifying true-negatives. This
effect is significantly greater as the inaccuracy of screening and confirmation tests increase.
Figure 2-19 Competing effect of number of screening tests on overall sensitivity versus specificity, and the role of convergence to bounds, where (a) both system bounds = 0.95 and (b) both system bounds = 0.80
(p = 0.15, r1 = 0.40, r2 = 0.10; both system bounds = 1–αc = 1–βc)
0.60
0.70
0.80
0.90
1.00
0 1 2 3 4 5 6 7 8 9 10 11
Prob
abili
ty o
f a c
orre
ct d
iagn
osis
Number of screening tests (j)
(a) Both system bounds = 0.95
1–αc = 1–βc = 0.95
0.60
0.70
0.80
0.90
1.00
0 1 2 3 4 5 6 7 8 9 10 11
Prob
abili
ty o
f a c
orre
ct d
iagn
osis
Number of screening tests (j)
(b) Both system bounds = 0.80
1–αc = 1–βc = 0.80
Case 1: 0.95 sensitivity Case 2: 0.75 sensitivity Case 3: 0.60 sensitivityCase 4: 0.95 specificity Case 5: 0.75 specificity Case 6: 0.60 specificity
Screening test accuracy:
Chapter 2 - Sequential Screening Models
58
The expected cost model also can be used to evaluate a variety of screening policies for any set
of values for test accuracies (αs, αc, βs, βc), test and false-diagnosis costs (ks, kc, kFP, kFN),
rescreening rates (r1, r2), incidence rate (p), and number of patients to be screened (n). Moreover,
the optimal number of screening tests (j*) which minimizes the expected total cost can be found,
such as by a simple search algorithm or more advanced optimization methods. In order to
illustrate this use and the significant sensitivity of the optimal policy to input assumptions,
Figure 2-20 compares the expected total costs for three different scenarios based on test
accuracies where the following values were assumed: ks = $5, kc = $25, kFP = $750, kFN = $5000,
r1 = 0.40, r2 = 0.10, p = 0.15, and n = 100.
Figure 2-20 Comparison of expected total cost per 100 patients (ks = $5, kc = $25, kFP = $750, kFN = $5000, r1 = 0.40, r2 = 0.10, p = 0.15)
The above examples help illustrate the general approach and explore how the overall system is
affected by the changes in various input assumptions. The figures used in these examples,
however, are only for illustrative purposes and do not reflect the actual practice. Additional
research is needed, for example, to better identify the screening and confirmation tests that can
be used appropriately in the sequential screening process and then to estimate their accuracies
$0
$5
$10
$15
$20
$25
$30
$35
$40
$45
0 1 2 3 4 5 6 7 8 9 10 11
Expe
cted
tota
l cos
t (in
thou
sand
s)
Number of screening tests (j)
Scenario 1: 0.95 sensitivity and specificity
Scenario 2: 0.60 sensitivity, 0.95 specificity
Scenario 3: 0.95 sensitivity, 0.60 specificity
j1* = 2
j2* = 4
j3* = 1
Screening and confirmation test accuricies:
Chapter 2 - Sequential Screening Models
59
and costs. Furthermore, a comprehensive sensitivity analysis, such as presented in Section 2.4,
should be conducted especially for the inputs that are subject to considerable uncertainty and
subjective discussion (e.g., false diagnosis costs).
2.6 Discussion
The design of any screening protocol should depend on the ability to detect truly positive and
truly negative patients and all costs incurred as a result of this protocol. The mathematical
models presented in this chapter can help policy makers, clinicians, and researchers understand
these characteristics for various possible protocols in order to design more effective and
minimum cost screening procedures for any particular situation. While the main focus of this
study is PTSD, a focus on optimally sequencing screening tests is likely warranted in a range of
other mental health disorders, such as illustrated in Section 2.5.
Given that untreated and undertreated PTSD result in significant physical and social problems as
well as tremendous medical and social costs, a well-designed and effective screening program is
vital in properly identifying and treating PTSD patients. This study investigated the current
screening processes and some alternatives for screening PTSD within the VHA and further
explored the effects of annual screenings, patient compliance, and prevalence on the expected
values, variances, and probability distributions of false-negatives, false-positives, and incurring
costs. Results indicate that a more intentionally designed system, which consists of a series of
annual screening tests along with a standardized confirmatory testing, results in lower false
diagnosis rates, predictable performance, and reduced costs. More specifically, an average of
55.07% and 18.10% decrease in the number of false-negative and false-positive diagnoses along
Chapter 2 - Sequential Screening Models
60
with a savings of 32.96% in total cost per year were shown to be possible by performing a
standardized confirmatory testing (with the use of PCL and CAPS) to follow-up the positive
results of the annual PC-PTSD screenings. These results, however, are very dependent on the
model inputs. A scenario analysis, therefore, was performed in order to illustrate how the overall
results are affected by various values of several model parameters, such as screening test
sensitivities and specificities, false-diagnosis costs, patient compliance rates, and prevalence.
This analysis also is useful to help decision makers explore the tradeoffs between annual
screenings and false-negative rates, false-positive rates, and costs.
One of the possible limitations of this study is that the most of the model inputs are not known
with certainty. This is especially true with respect to the costs of false-negative and false-positive
results and patient compliance rates. Although all input estimations were based on discussions
with several clinicians who have a thorough knowledge of PTSD and a sensitivity analysis was
performed for a range of possible values, additional research could be useful to confirm these
estimates or to better estimate the aforementioned inputs. Another limitation is the assumption
that the VA has the capacity to perform the confirmatory screening and to provide the treatment
that would be recommended after the screening.
Chapter 3 - Multi-State Categorical Diagnostic Models
61
Multi-State Categorical Diagnostic Models Chapter 3 -
Traumatic brain injury (TBI), one of the most common military injuries, is an enormous problem
facing most U.S. soldiers today. Given that the vast majority of TBI diagnoses among military
personnel are categorized as mild, many cases go undetected and untreated, resulting in
significant psychological disorders, long-term disabilities, and economic burdens. Assessment
tools for TBI, which are found to be efficient when the extent of the injury is more severe, often
are insufficient when there are no obvious signs of trauma. This chapter presents several
predictive diagnostic models to determine whether or not an individual has TBI and to categorize
him into the most likely severity state. These models aim to improve the reliability and accuracy
of TBI diagnoses by combining the results of different screening tests, which are conducted
consecutively over time, with the use of several mathematical techniques, namely, fuzzy logic,
logistic regression, and neural networks.
3.1 Background
Military personnel and others working in combat zones are at particular risk of traumatic brain
injury (TBI) due to the fact that they are likely to be exposed to events that may cause TBI, such
as concussive blasts or explosions (Champion et al., 2009; Ling et al., 2009; Defense and
Veterans Brain Injury Center, 2013). In combat zones, TBI often occurs in tandem with
Chapter 3 - Multi-State Categorical Diagnostic Models
62
ostensibly life-threatening injuries; therefore, cases may go unrecognized, potentially putting the
individuals or their units at risk (Butler et al., 2008; Tanielian and Jaycox, 2008). Additionally,
when TBI occurs with no outward signs of trauma (i.e., mild cases), service members might not
seek medical treatment (Martin et al., 2008; McCrea, 2008). Many TBI sufferers, especially if
untreated, may experience medical, behavioral, and social problems for a long time, perhaps a
lifetime. Any delays in treatment may compromise recovery or result in serious cognitive,
physical, and/or psychological problems and even permanent damage (Rao and Lyketsos, 2000;
Maas et al., 2008; Anderson-Barnes et al., 2010; Faul et al., 2010).
The existence and severity of TBI are evaluated in several ways, including subjective
assessments, external signs, physical experiences, neurologic assessments (e.g., the Glasgow
Coma Scores (GCS)), and clinical assessments (e.g., computed tomography (CT) scans,
magnetic resonance imaging (MRI), and electroencephalogram (EEG)) (U.S. Department of
Health and Human Services, 2006; Elder et al., 2010; Maruta et al., 2010). In the military,
testing typically follows a process of screening, identification, evaluation, confirmation, and
stratification by severity (Butler et al., 2008). While ideally all TBI would be detected soon after
the time of injury, initial tests tend to be less accurate than more thorough and expensive post-
deployment tests. GCS, developed by Teasdale and Jennett (1974), for instance, is a valuable
first assessment of TBI severity and prognosis immediately after injury. However, it has several
practical limitations (Wijdicks et al., 2005; Matis and Birbilis, 2008; Zuercher et al., 2009;
Middleton, 2012), generally due to the fact that often it is not measured in emergency rooms or
in hospitals where TBI patients are first transported (Thatcher et al., 2001). Conventional CT and
MRI, the most frequent imaging modalities used in acute diagnosis and management, often
cannot detect mild TBIs since generally the brain appears quite normal (Dhawan et al., 2006;
Chapter 3 - Multi-State Categorical Diagnostic Models
63
Shenton et al., 2012). EEG, on the other hand, is recommended to detect mild TBI even though it
is not considered sensitive enough to distinguish mild and moderate TBIs (Thatcher, 2000;
Haneef et al., 2013). In practice more than one method can be used for diagnosis to assure
reliability and accuracy. Guler et al. (2008), for instance, developed a diagnostic system for
determining the severity of civilian TBI by using fuzzy logic to combine information from GCS
and EEG results. They found a significant relationship between the findings of neurologists and
the systems output for normal, mild, and severe electroencephalography tracing data. In the
study, fuzzy inferences were developed from GCS and EEG trauma score data, which were then
used to produce a trauma severity degree and final diagnosis for each patient. However, neither
an analysis on the accuracy or optimization of their approach nor a discussion on the longitudinal
nature of increasingly accurate data was presented.
This study proposes a predictive modeling approach using fuzzy logic, multinomial logistic
regression, and neural networks, which provides a comprehensive screening procedure as
illustrated in Figure 3-1. The main concern of this procedure is to better identify mild cases
which occur without any visible symptoms and to prevent any delays in treatment. Also, in the
proposed procedure, the determination can change continuously as each screening test result
becomes available, presumably getting more accurate over time, given that the symptoms of mild
injuries may not be immediately recognized.
Chapter 3 - Multi-State Categorical Diagnostic Models
64
Figure 3-1 General illustration of longitudinal TBI classification model
3.2 Fuzzy Categorical Model
3.2.1 Methodology
Fuzzy logic (FL), which was first introduced by Lotfi A. Zadeh (1965), is a computational
paradigm for processing information in a way that resembles human reasoning in the presence of
uncertainty (Ross, 1995). It has been successfully used in healthcare applications, including
Veryha and Adamczyk (2005) in primary dental care services, Dalalah and Magableh (2008) for
remote health service delivery, Guler et al. (2008) for detecting civilian TBI severity, Khan et al.
(2008) for cancer prognosis, Medjahed et al. (2012) for remote healthcare monitoring, Torshabi
et al. (2012) for tumor motion prediction, Czabanski et al. (2013) for classification of fetal heart
rate tracings, and Kwiatkowska and Kielan (2013) for assessment of clinical depression.
Fuzzy logic emulates a person’s ability to categorize things by assessing the degree of
membership rather than evaluating whether they satisfy some unambiguous definition (Lootsma,
1997). A fuzzy variable (here TBI severity categories, i.e., normal, mild, moderate, and severe) is
characterized by its name tag, a set of fuzzy values (also known as linguistic values or labels),
t1 t2 t3 tN Time
Screening test N result: XN
Screening test 3 result: X3
Screening test 2 result: X2
Screening test 1 result: X1
Determination for n=3
Inference Phase
Determination for n=N
Determination for n=1
Determination for n=2
Chapter 3 - Multi-State Categorical Diagnostic Models
65
and the membership function of these labels. These membership functions assign a membership
value, μLabel(x), to a given real value x within some predefined range, R (known as the universe
of discourse). Two basic fuzzy operations, “and” and “or”, are defined as (Guler et al., 2002):
Definition 1: μA and B(x) = 𝜇A(x) ∧ μB(x) = min�μA(x), μB(x)�
and
Definition 2: μA or B(x) = 𝜇A(x) ∨ μB(x) = max�μA(x), μB(x)�,
where A and B are fuzzy variables. Using these operators, fuzzy variables can be combined to
form fuzzy-logic expressions.
The general fuzzy logic algorithm consists of three activities: fuzzification, fuzzy inference, and
defuzzification. Fuzzification is the process of making a crisp numeric quantity fuzzy (Ross,
1995) by mapping it onto fuzzy membership functions. Typically, conventional trapezoid shapes,
such as those shown in Figure 3-2, are used to calculate fuzzy membership functions or degrees
of membership for each category as
μ(xi) =
⎩⎪⎨
⎪⎧
xi−ab−a
, a ≤ xi ≤ b1, b < xi < cd−xid−c
, c ≤ xi ≤ d0, otherwise
, (3-1)
where a, b, c, and d are the defining parameters of each membership function.
Chapter 3 - Multi-State Categorical Diagnostic Models
66
Figure 3-2 Fuzzy membership functions for trauma severity degrees
The second step, fuzzy inference, is the process of obtaining a fuzzy output using a rule-base that
combines several inputs (here several screening tests), i.e., the results of the above fuzzification
linguistic inputs. The rules in the rule-base define the connection between input and output fuzzy
variables (Guler et al., 2002). The relationships and outputs obtained from the rule-base are
interpreted using “and” and “or” operators (Guler et al., 2008).
Finally, defuzzification is the conversion of these fuzzy quantities to a crisp value z*. Several
methods exist for combining and defuzzifying fuzzy membership values, including the max-
membership, center of gravity (COG), weighted average (WA), mean-of-maxima membership
(MOM), center of sums (COS), center of largest (COL) area, and smallest or largest of maxima
(SOM or LOM) methods (Ross, 1995). By example, maxima methods choose the set with the
maximum membership, whereas in the COG method, a common and useful approach, the center
of gravity of the area under the membership functions is calculated as
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
1.10
Degr
ee o
f mem
bers
hip
Trauma degree
NormalMildModerateSevere
aM bM cM dM
Chapter 3 - Multi-State Categorical Diagnostic Models
67
z* =∫ μC(x) ∙ x dx
∫ μC(x) dx, (3-2)
where C is the union of the membership functions. First, the results of the rules are added
together and the top parts of the graphs are chopped off to form trapezoids. All of these
trapezoids are then superimposed one over another, forming a single geometric shape. The
centroid of this shape, called the fuzzy centroid, is then calculated and its x coordinate gives the
defuzzified value.
As an extension to Guler et al. (2008), a more comprehensive procedure was developed by
sequentially applying fuzzy inference longitudinally to a series of screening results as they
become available. Applying a fuzzy diagnostic system sequentially N times (i.e., to N screening
test results) eventually produces N trauma severity degree values (X1, X2, …, XN) for each
patient, which become the inputs to the fuzzification logic. Without the loss of generality, these
values were assumed to be between 0 and 100 based on the expert-based scale from Guler et al.
(2008). Four fuzzy sets – normal (no TBI), mild, moderate, and severe – were formed for these
values using the four trapezoid membership functions shown in Figure 3-2 (which are different
for each screening test, resulting in 4N membership functions). Note that the parameters of each
membership function (ai, bi, ci, and di for fuzzy set i, where i = No(rmal), Mi(ld), Mo(derate),
and Se(vere)) typically might be determined by experts or with an optimization tuning routine,
such as demonstrated later. Membership degrees for each category (i.e., for each of the four
trauma severities) were then computed using Eq. (3-1).
Chapter 3 - Multi-State Categorical Diagnostic Models
68
For the fuzzy inference step, a rule-base which consists of four linguistic outputs – normal, mild,
moderate, and severe – the same as the membership functions was used. Table 3-1 illustrates the
rule-base for the case with two screening results (N = 2). As seen, after the second screening
there are 42 = 16 rules. To illustrate interpretation of this rule-base, if the output of screening test
1 is “mild” (column 2) and the output of screening test 2 is “moderate” (row 3), then the output
of the fuzzy inference process is “moderate” (shown in italic font). According to the rule-base
and using min-max operators, the fuzzy outputs of the inference process were obtained and
represented by μis(Xs), which means that according to the result Xs of screening test s, a patient
belongs to fuzzy set i with a membership degree of μis evaluated at the value Xs, where s = 1, 2,
…, N and i = No, Mi, Mo, Se. These fuzzy values were then converted into crisp values using the
defuzzification methods mentioned above.
Table 3-1 Example of a rule-base for N = 2 screening test case
The output of the inference Output of screening test 1
Normal Mild Moderate Severe
Output of screening
test 2
Normal Normal Mild Mild Mild Mild Mild Mild Moderate Moderate
Moderate Moderate Moderate Moderate Severe Severe Moderate Severe Severe Severe
3.2.2 Numerical Example
Consider the N = 4 case with four screening tests, where the parameters of each membership
function are given as aNo = bNo = 0, cNo = 15, dNo = 40, aMi = 15, bMi = 30, cMi = 40, dMi = 50,
aMo= 30, bMo = cMo = 50, dMo = 75, aSe = 50, bSe = 70, and cSe = 80. For simplicity, the
membership functions were assumed to have the same parameters for all four tests, although in
Chapter 3 - Multi-State Categorical Diagnostic Models
69
reality they likely would be different. The fuzzy categorical model was applied to categorize a
patient with the trauma severity degrees of X1 = 45, X2 = 56, X3 = 65, and X4 = 20 obtained from
the screening tests. According to the fuzzy membership functions and using Eq. (3-1), the
membership degrees for each TBI category according to the screening test 1 were calculated as
μMi1 (45) =
dMi − X1
dMi − cMi=
50 − 4550 − 40
= 0.50
and
μMo1 (45) =
X1 − aMo
bMo − aMo=
45 − 3050 − 30
= 0.75.
Similarly, the membership degrees for each category according to the screening tests 2, 3, and 4
were calculated as μMo2 (56) = 0.76 and μSe
2 (56) = 0.30, μMo3 (65) = 0.48 and μSe
3 (65) = 0.65, and
μNo4 (20) = 0.80 and μMi
4 (20) = 0.33.
With four tests, now there are 44 = 256 rules, although only 24 = 16 of these rules are needed in
this example (listed in Table 3-2) since each trauma severity degree has a membership degree
different than zero for only two of the categories. Using this rule-base and min-max operators,
this patient was determined to belong to the “mild” and “moderate” fuzzy sets with membership
degrees from rules 1-2 and rules 3-16, respectively, of
μMi = max �min �μMi1 (X1),…,μNo
4 (X4)� , min �μMi1 (X1),…,μMi
4 (X4)��
= max{min(0.5, 0.76, 0.48, 0.80), min(0.5, 0.76, 0.48, 0.33)} = 0.48
Chapter 3 - Multi-State Categorical Diagnostic Models
70
and
μMo = max �min �μMi1 (X1),…,μNo
4 (X4)� , min �μMo1 (X1),…,μMi
4 (X4)��
= max{min(0.5, 0.76, 0.65, 0.80), min(0.75, 0.30, 0.65, 0.33)} = 0.65.
Table 3-2 Rule-base for N = 4 screening test example
Rule Output of scr. test 1
Output of scr. test 2
Output of scr. test 3
Output of scr. test 4
Output of inference
1 Mild Moderate Moderate Normal Mild 2 Mild Moderate Moderate Mild Mild 3 Mild Moderate Severe Normal Moderate 4 Mild Moderate Severe Mild Moderate 5 Mild Severe Moderate Normal Moderate 6 Mild Severe Moderate Mild Moderate 7 Mild Severe Severe Normal Moderate 8 Mild Severe Severe Mild Moderate 9 Moderate Moderate Moderate Normal Moderate 10 Moderate Moderate Moderate Mild Moderate 11 Moderate Moderate Severe Normal Moderate 12 Moderate Moderate Severe Mild Moderate 13 Moderate Severe Moderate Normal Moderate 14 Moderate Severe Moderate Mild Moderate 15 Moderate Severe Severe Normal Moderate 16 Moderate Severe Severe Mild Moderate
Next, these fuzzy values were converted into a crisp value using any of the defuzzification
methods mentioned above. The present study compared the results of four methods – COG,
MOM, SOM, and LOM. Using the COG (see Eq. (3-2)) and MOM methods, the final trauma
severity degrees were calculated to be 44.75 (shown below) and 50.88, respectively, both of
which indicate a final diagnosis of “moderate trauma” using the expert-based scale from Guler et
al. (2008). Using the SOM and LOM methods, however, the final trauma severity degrees were
43.00 and 58.75, respectively, corresponding to “moderate” and “severe” trauma on this scale.
Chapter 3 - Multi-State Categorical Diagnostic Models
71
QfinalCOG =
∫ � x15−1�xdx22.2
15 +∫ 0.48xdx39.622.2 +∫ � x
20−32�xdx43
39.6 +∫ 0.65xdx58.7543 +∫ �− x
25+3�xdx7558.75
∫ � x15−1�dx22.2
15 +∫ 0.48dx39.622.2 +∫ � x
20−32�dx43
39.6 +∫ 0.65xdx58.7543 +∫ �− x
25+3�dx7558.75
= 44.75.
Table 3-3 illustrates how the final diagnosis changes over time as the result of each screening test
is incorporated and as each defuzzification method is used. To automate this process, the
predicted health states for each patient can be determined using MATLAB’s fuzzy logic tool.
Table 3-3 Changes in diagnosis over time and according to defuzzification method
Time Screening test 1
Screening test 2
Screening test 3
Screening test 4
Final diagnosis COG MOM SOM LOM
t1 45 (not yet available) (not yet available) (not yet available) Moderate Moderate Moderate Severe t2 45 56 (not yet available) (not yet available) Severe Moderate Moderate Severe t3 45 56 63 (not yet available) Severe Moderate Moderate Severe t4 45 56 63 20 Moderate Moderate Moderate Severe
3.2.3 Model Accuracy
The predicted diagnoses for a large cohort of patients were compared to their ultimate known
states which were determined via expert consensus. Table 3-4 illustrates a portion of a
hypothetical dataset with 100 patients. Using the N = 4 case with four screening tests and
membership functions from the previous section, Figure 3-3 summarizes the correlation between
the actual and predicted TBI states. 77% of all results are correctly classified using all 4 tests,
with a correlation coefficient of R = 0.837047, indicating a fairly strong agreement.
Chapter 3 - Multi-State Categorical Diagnostic Models
72
Table 3-4 Illustrative portion of evaluation dataset
Patient #
Actual health state
Result of scr. test 1 (X1)
Result of scr. test 2 (X2)
Result of scr. test 3 (X3)
Result of scr. test 4 (X4)
1 4 54 84 78 51 2 2 70 84 44 21 3 4 52 68 45 79 4 3 50 79 2 46 5 2 10 18 33 54 6 4 80 61 64 47 7 3 77 36 44 54 8 2 39 41 35 11 9 2 10 24 12 10 10 3 69 7 21 40 11 2 20 42 34 9 12 4 50 84 65 41 13 3 49 17 20 25 14 4 81 75 86 88 15 4 54 64 74 53 …
…
…
…
…
…
Four categories for the health state of a patient: 1, 2, 3, and 4, representing “Normal”, “Mild”, “Moderate”, and “Severe” states, respectively
Figure 3-3 Accuracy of fuzzy categorical model results over time (The larger the bubbles, the more accurate the estimations)
0
1
2
3
4
5
0 1 2 3 4 5
Pred
icte
d st
ate
Actual state
Time t1
R = 0.724168
0
1
2
3
4
5
0 1 2 3 4 5
Pred
icte
d st
ate
Actual state
Time t2
R = 0.766746
0
1
2
3
4
5
0 1 2 3 4 5
Pred
icte
d st
ate
Actual state
Time t3
R = 0.790136
0
1
2
3
4
5
0 1 2 3 4 5
Pred
icte
d st
ate
Actual state
Time t4
R = 0.837047
Chapter 3 - Multi-State Categorical Diagnostic Models
73
3.3 Multinomial Logistic Regression Categorical Model
3.3.1 Methodology
Multinomial logistic regression (MLR) generalizes logistic regression to allow more than two
discrete outcomes (i.e., categories) for the dependent variable (Wang, 2005). Its performance has
been extensively discussed in the healthcare literature, including these by Haas-Wilson and
Savoca (1990), Hossain et al. (2002), Peng and Nichols (2003), Zhu et al. (2010), and
Upadhyaya et al. (2013).
MLR method builds a separate equation for each category and each equation provides a
predicted probability of being in the corresponding or any lower category (Sentas et al., 2005).
Here, it was used to categorize patients into the most probable state of severity by predicting
cumulative probabilities for each severity state. The prediction equations are of the general form
l�πj� = b0(j) + ∑ bi
(j)xiNi=1 , (3-3)
where πj is the cumulative probability for the jth category, the b0(j) and bi
(j) terms are the
regression coefficients for the jth category, xi is the ith predictor variable (test result), and N is
the number of tests. After estimating the model coefficients via maximum likelihood, the
cumulative probability for each category j is calculated by solving Eq. (3-3) for πj. Non-
cumulative probabilities are calculated via subtracting, i.e., pj = πj – πj–1, and the category with
maximum probability is selected as the final diagnosis (Norris et al., 2006; Sentas and Angelis,
2006; Asgary et al., 2007). As with any logistic regression, the model predicts a transformation
Chapter 3 - Multi-State Categorical Diagnostic Models
74
l(πj) of the desired probabilities, rather than the probabilities themselves. The function l(.) is
called link function (Sentas et al., 2005), with the most commonly used link functions being
− a logit function, log� πj
1−πj�, used for evenly distributed categories,
− a complementary log-log function, log�−log�1− πj��, used when higher categories are more probable,
− a log-log function, −log �−log�πj��, used when lower categories are more probable, and
− a probit function, ϕ−1�πj� where ϕ is the Gaussian cumulative probability density, used when latent variable is normally distributed (Chan, 2005).
As shown in Figure 3-4, these link functions differ in their degree of agreement depending on the
value of π.
Figure 3-4 A graphical comparison of link functions
Since all categories are evenly distributed, i.e., none of the categories are more probable, here the
logit function was used as the link function,
logit
probit
comploglog
loglog
-6
-4
-2
0
2
4
6
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1π
l (π)
Chapter 3 - Multi-State Categorical Diagnostic Models
75
log� πj
1−πj� = b0
(j) + ∑ bi(j)xi
Ni=1 , (3-4)
where j = 1, 2, …, q–1. Thus the exact probabilities for each category are in the form of
pj =eb0
(j)+∑ bi(j)xi
ni=1
1 + eb0(j)+∑ bi
(j)xini=1
−eb0
(j−1)+∑ bi(j−1)xi
ni=1
1 + eb0(j−1)+∑ bi
(j−1)xini=1
, (3-5)
where j = 1, 2, …, q–1, and
pq = 1 −� pj
q−1
j=1
. (3-6)
3.3.2 Numerical Example
Again using the N = 4 screening tests case and the same dataset from the previous section, the
MLR coefficients were calculated via MINITAB to produce
log� π11−π1
� = 1.117 − 0.062Q1 − 0.036Q2 − 0.046Q3 − 0.035Q4,
log� π21−π2
� = 5.757 − 0.062Q1 − 0.036Q2 − 0.046Q3 − 0.035Q4,
and
log� π31−π3
� = 9.791 − 0.062Q1 − 0.036Q2 − 0.046Q3 − 0.035Q4.
Chapter 3 - Multi-State Categorical Diagnostic Models
76
By solving these equations, the cumulative probabilities were calculated and then with
subtraction, the probabilities p1, p2, p3, and p4 were obtained. By example, for the same patient in
Section 3.2.2 the probability of the category 1 (Normal) was calculated as
p1 = π1 =e1.117−0.062(45)−0.036(56)−0.046(63)−0.035(20)
1 + e1.117−0.062(45)−0.036(56)−0.046(63)−0.035(20) = 0.0006.
Similarly, the probabilities of the category 2, 3, and 4 were calculated to be p2 = π2 – π1 = 0.0636,
p3 = π3 – π2 = 0.7307, and p4 = 1 – ∑ pj3j=1 = 0.2050, respectively. Since the probability of
category 3 is the greatest, this patient was given a final diagnosis of “Moderate” (i.e., category
3).
3.3.3 Model Accuracy
Using the same evaluation data set as in Section 3.2.3, Table 3-5 summarizes the category
probabilities for the first 15 patients. Figure 3-5 compares the actual and predicted states over
time, with 73% of patients being correctly classified after all four tests and a correlation
coefficient of R = 0.799164 nearly as strong as the fuzzy categorical model.
Chapter 3 - Multi-State Categorical Diagnostic Models
77
Table 3-5 Category probabilities for MLR approach
Patient #
Prob. of category 1 (p1)
Prob. of category 2 (p2)
Prob. of category 3 (p3)
Prob. of category 4 (p4)
Final classification
1 0.000023 0.002383 0.117400 0.880194 4 2 0.000118 0.011911 0.395217 0.592755 4 3 0.000081 0.008261 0.313562 0.678095 4 4 0.001416 0.126670 0.764269 0.107645 3 5 0.027531 0.718158 0.248304 0.006007 2 6 0.000023 0.002385 0.117503 0.880089 4 7 0.000137 0.013859 0.430769 0.555234 4 8 0.008200 0.453112 0.518414 0.020274 3 9 0.217870 0.748631 0.032886 0.000614 2
10 0.003030 0.236395 0.707282 0.053293 3 11 0.028387 0.723232 0.242559 0.005822 2 12 0.000077 0.007847 0.302782 0.689293 4 13 0.012815 0.560656 0.413521 0.013009 2 14 0.000001 0.000117 0.006517 0.993365 4 15 0.000054 0.005503 0.234165 0.760278 4
…
…
…
…
…
…
Four categories for the health state of a patient: 1, 2, 3, and 4, representing “Normal”, “Mild”, “Moderate”, and “Severe” states, respectively
Figure 3-5 Accuracy of MLR categorical model results over time (The larger the bubbles, the more accurate the estimations)
0
1
2
3
4
5
0 1 2 3 4 5
Pred
icte
d st
ate
Actual state
Time t1
R = 0.724168
0
1
2
3
4
5
0 1 2 3 4 5
Pred
icte
d st
ate
Actual state
Time t2
R = 0.766357
0
1
2
3
4
5
0 1 2 3 4 5
Pred
icte
d st
ate
Actual state
Time t3
R = 0.737442
0
1
2
3
4
5
0 1 2 3 4 5
Pred
icte
d st
ate
Actual state
Time t4
R = 0.799164
Chapter 3 - Multi-State Categorical Diagnostic Models
78
3.4 Artificial Neural Network Model
3.4.1 Methodology
As a final approach, an artificial neural network (ANN) can be used to categorize patients into
their most probable state of severity. ANN is a non-linear mathematical model that mimics the
structure or functional aspects of biological neural networks. Its structure development is based
on information that flows through it during a learning phase (Guler et al., 2009). ANNs have
been successively used as decision support tools for the early detection, diagnosis, and prognosis
of a wide range of diseases (Vaughn et al., 2000; Mangiameli et al., 2004; Lisboa and Taktak,
2006), such as blunt trauma (Marble and Healy, 1999), breast cancer (West and West, 2000;
West et al., 2005), prostate cancer (Anagnostou et al., 2003; Remzi and Djavan, 2004; Cinar et
al., 2009), skin diseases (Chang and Chen, 2009), traumatic brain injury (Guler et al., 2009), and
liver disease (Lin and Chuang, 2010).
An ANN consists of an input layer, an output layer, and a number of hidden layers, with each
layer containing a predetermined number of artificial neurons (see Figure 3-6). Each neuron
receives an input from an upstream neuron and sends an output signal to a downstream neuron,
with connections between neurons optimally weighted by a learning algorithm to create a desired
final output.
Chapter 3 - Multi-State Categorical Diagnostic Models
79
Figure 3-6 Generic structure of an ANN
The strength of the connection between an input k and a neuron j is noted by the value of its
weight wkj. To model the actual activity inside a neuron (see Figure 3-7), an adder sums up all
the inputs modified by their respective weights as a linear combination. An activation function
φ(.) then controls the amplitude of the neuron’s output, with an acceptable output range usually
being between 0 and 1 or –1 and 1. The activity of a neuron is written as
oj = φ�netj� = φ�∑ wkjxjnk=1 �, (3-7)
where oj is neuron j’s output, and netj is the summation of the weighted inputs received by
neuron j.
wih
Input Hidden Output
who
Chapter 3 - Multi-State Categorical Diagnostic Models
80
Figure 3-7 Illustration of general input-output neuron activity
The common activation functions include two soft non-linearity activation functions,
− a sigmoid function, oj = 11+e−netj, and
− a hyperbolic tangent function, oj = tanh �netj2�;
along with two hard non-linearity activation functions,
− a signum function, oj = �1,
undefined,−1,
netj > θjnetj = θjnetj < θj
, and
− a step function, oj = �1,0,
undefined,
netj > θjnetj = θjnetj < θj
(Krose and Smagt, 1996).
Inputs Weights
Transfer function
Net input
Activation function
Threshold
Activation
Chapter 3 - Multi-State Categorical Diagnostic Models
81
3.4.2 Numerical Example
A four-layered Multilayer Perceptron (MLP) with two hidden layers is one of the most widely
implemented neural network topologies and is a universal pattern classifier (Sut and Senocak,
2007). In this example, such an MLP which is trained with a back-propagation learning
algorithm was used to classify the patients into the most probable state of severity. Considering
the N = 4 screening tests case and the same data set, the input layer had four input nodes
identifying each test and the Tan-hAxon transfer function was used in the nodes of the hidden
and output layers of the ANN. The training data had 51 samples. After the network was
established via Neurosolutions, multiple trainings were conducted and the best one (with the
minimum mean square error (MSE)) was used. Thus, the same patient in Section 3.2.2 was given
a final diagnosis of “Moderate”.
Figure 3-8 ANN network established in Neurosolutions for N = 4 screening test example
3.4.3 Model Accuracy
Using the same evaluation data set as above, Figure 3-9 compares the actual and predicted states
over time with 81% of patients correctly classified after all 4 tests and a correlation coefficient of
R = 0.848659, which is slightly better than the fuzzy and MLR categorical models.
Chapter 3 - Multi-State Categorical Diagnostic Models
82
Figure 3-9 Accuracy of ANN categorical model results over time (The larger the bubbles, the more accurate the estimations)
3.5 Model Comparison and Optimization
Table 3-6 and 3-7 summarize the results of all three approaches, indicating that each method
provides a fairly strong agreement with patients’ known health states.
Table 3-6 Changes in final diagnosis over time based on the method used
Time Screening test 1
Screening test 2
Screening test 3
Screening test 4
Final diagnosis FL OLR ANN
t1 45 N/A N/A N/A Moderate Moderate Moderate t2 45 56 N/A N/A Severe Moderate Severe t3 45 56 63 N/A Severe Severe Severe t4 45 56 63 20 Moderate Moderate Moderate
FL: Fuzzy logic, MLR: Multinomial logistic regression, ANN: Artificial neural networks
0
1
2
3
4
5
0 1 2 3 4 5
Pred
icte
d st
ate
Actual state
Time t1
R = 0.724168
0
1
2
3
4
5
0 1 2 3 4 5
Pred
icte
d st
ate
Actual state
Time t2
R = 0.799270
0
1
2
3
4
5
0 1 2 3 4 5
Pred
icte
d st
ate
Actual state
Time t3
R = 0.825036
0
1
2
3
4
5
0 1 2 3 4 5
Pred
icte
d st
ate
Actual state
Time t4
R = 0.848659
Chapter 3 - Multi-State Categorical Diagnostic Models
83
Table 3-7 Overall comparison of model performances
Method Correct diagnosis % MSE Correlation
coefficient (R) FL 77 0.26 0.837047
MLR 73 0.30 0.799164 ANN 81 0.25 0.848659
FL: Fuzzy logic, MLR: Multinomial logistic regression, ANN: Artificial neural networks
In the fuzzy categorical model, intermediate and final diagnoses may be fairly dependent on all
4N membership function shapes and 16N parameters (a, b, c, d), N intermediate and final rule-
bases, and N defuzzification scales (all potentially decision variables). Thus, model performance
could be improved or optimized by searching out the values of some or all of the above decision
variables to optimize each of the above performance criteria and to minimize the number of
undetected or misdiagnosed cases. Several models could be developed to optimize membership
function values, rule-bases, and defuzzification scales – the optimal values of which might be
different for each intermediate and final number of screening tests (N). For example, an optimal
rule-base might be developed using a quadratic mathematical model that minimizes the square
errors, as shown in Eq.(3-8); even in this simple case the number of decision variables may be as
many as 4n.
Minimize z = ∑ �xkl − rj�2
�j|O1j=k,O2j=l� ∀ k, l
Subject to
xkl ≤ 4
xkl ≥ 1 and integer. (3-8)
The different solutions that are based on different criteria also can be weighted or traded-off
against one another using various multi-criteria or desirability approaches. Note that, depending
Chapter 3 - Multi-State Categorical Diagnostic Models
84
on how exhaustive the model is, the number of decision variables may grow exponentially and
special heuristics may be necessary for finding optima.
3.6 Discussion
TBI is a major military, civilian, and public health concern and one of the leading causes of
disability and death in the U.S. Although improvements in protective equipment and battlefield
care have reduced the incidence of penetrating head injuries and death among military personnel,
there has been a significant increase in the incidence of closed-head TBI, which often occurs
without any visible symptoms and goes unrecognized and untreated indefinitely. Proper and
timely diagnosis is essential in effectively treating a TBI and preventing any medical, behavioral,
and social consequences that untreated patients may experience. Currently available methods to
assess the existence and severity of TBI may be insufficient, especially in detecting mild cases.
Additionally, the initial categorization of a TBI does not necessarily correspond to the ultimate
outcome as symptoms may get worse over time, eventually leading to severe dysfunction and
disability.
This study presents a comprehensive screening procedure, which combines different assessment
tools and consists of multiple screenings over time, in order to reduce the number of
unrecognized and misdiagnosed cases. Three types of mathematical methods – fuzzy logic,
multinomial logistic regression, and neural networks – were used to categorize TBI patients into
the one of four health states, namely, none, mild, moderate, or severe. A numerical example was
developed for illustration purposes and preliminary results for each method indicated a fairly
strong agreement with the patients’ known health states.
Chapter 3 - Multi-State Categorical Diagnostic Models
85
Future work should be conducted to further develop the proposed modeling framework, which
includes identifying the appropriate screening tests and their characteristics, determining the
predicted diagnoses for patients whose actual health states are known, and finally evaluating the
overall or severity-specific performance. Additional research could be useful to better identify
the model parameters, such as fuzzy logic membership functions, final rule-bases, and
defuzzification scales. Also, overall or severity-specific performance of the proposed model
could be optimized by developing several optimization models, such as illustrated in Section 3.5.
Chapter 4 - Network Optimization Models
86
Network Optimization Models Chapter 4 -
Optimal location of specialty care services and allocation of patients to these services within any
healthcare network are increasingly important for balancing costs, access to care, and patient-
centeredness. Typical long-range planning efforts attempt to address a myriad of quantitative and
qualitative issues, including within-network access within reasonable travel distances, space
capacity constraints, costs, politics, and community commitments. To help inform these
decisions, this chapter presents several mathematical integer programs that minimize total
procedure, travel, non-coverage, and start-up costs to increase network capacity subject to access
constraints. These models were used across a range of specialty care services within the Veterans
Health Administration (VHA) to help decision makers explore relationships and tradeoffs
between costs, coverage, service location, and capacity and to inform larger strategic planning
discussions.
4.1 Background
Over a decade ago, the Institute of Medicine (IOM) released two landmark reports outlining
significant deficiencies in the quality and safety of medical care provided in the United States. To
Err is Human (1999) documented the nearly 100,000 lives lost each year due to systematic
problems in the design and execution of healthcare delivery. Crossing the Quality Chasm (2001)
Chapter 4 - Network Optimization Models
87
provided a broad vision of how health care could be effectively transformed and improved, and
recommended the use of systems and industrial engineering techniques to assist with
understanding and optimizing healthcare processes. In response to this call, the National
Academy of Engineering (NAE) and the IOM initiated a study in 2002 to identify engineering
tools and technologies that could help the health system improve and deliver care that is safe,
effective, timely, patient-centered, efficient, and equitable – the six health system aims defined in
Crossing the Quality Chasm.
The result of this study was published as Building a Better Delivery System (National Academy
of Engineering and Institute of Medicine, 2005), which concluded that the U.S. healthcare
industry has neglected use of systems engineering methods and concepts that have revolutionized
quality, productivity, and performance in many other industries, and that this “collective
inattention” has contributed to serious consequences within health care – huge amounts of
preventable harm and deaths, outdated procedures, an approximately half-trillion dollars wasted
annually through inefficiency, costs rising at roughly three times the rate of inflation, and 43
million people uninsured. Today, health care is still an area of crucial concern in the United
States, spending substantially more than any other developed country in the Organization for
Economic Co-operation and Development (OECD). The use of healthcare services in the U.S.,
however, is below the OECD median by almost all measures, which means that Americans are
receiving fewer real resources than people in over 50% of OECD countries. These facts suggest
that the difference in spending is caused mostly by higher prices for the delivery of healthcare
goods and services in the U.S. (Anderson et al., 2003). Rising costs and inadequate levels of care
underscore the importance of making strategic decisions within this industry.
Chapter 4 - Network Optimization Models
88
The use of systems engineering models to help make decisions in many key areas, in fact, can be
very effective at helping design both cost-efficient and qualified healthcare services, especially
in cases involving complex and competing considerations. Network planning is one of several
areas that can be improved via systems engineering methods, with the optimal location of health
services across geographic care networks having significant potential to improve costs, patient-
centeredness, and care continuity (3 of the IOM dimensions).
Location-allocation models seek to simultaneously determine optimal facility locations and the
assignment of customers (here, patients) to facilities. These models have been applied widely in
many industries to reduce costs and increase access, with examples spanning retail,
telecommunication, energy, and manufacturing. In health care, poor location decisions also can
result in increased mortality and morbidity, with optimal facility location therefore having even
greater importance (Daskin and Dean, 2005). Historical healthcare applications have included
locating ambulances (Adenso-Díaz and Rodríguez, 1997; Sasaki et al., 2010) and hospitals in
rural regions (Mehrez et al., 1996), trauma care resources (Branas et al., 2000), organ transplant
services (Bruni et al., 2006), blood facilities (Jacobs et al., 1996), emergency medical service
vehicles (Eaton et al., 1985), hospital networks (Santibanez et al., 2009), and specialized care
services such as traumatic brain injury (TBI) treatment (Côté et al., 2007; Syam and Côté, 2010).
Models that account for transient populations who change their locations seasonally (Ndiaye and
Alfares, 2008) and continuously changing nature of health systems (Harper et al., 2005) also
have been developed for specific applications.
From a modeling perspective, location problems can be classified as discrete and continuous.
Discrete models assume a finite number of candidate locations exist at which facilities can be
Chapter 4 - Network Optimization Models
89
sited, whereas continuous models assume facilities can be located anywhere within a geographic
area. All models aim simultaneously to determine both the best locations for facilities and the
best assignment (“allocation”) of individuals to these facilities, where “best” usually is defined
by some combination of total travel distance, facility costs that include fixed locating and
variable operating costs, and demand coverage. The work in this chapter focuses on discrete
location models, which also have been used more extensively in healthcare location problems
(Daskin and Dean, 2005), because typically the VHA has been interested in considering specific
existing or candidate locations. Table 4-1 summarizes the three basic types of discrete facility
location models and the performance measures they seek to optimize.
Table 4-1 Representation of basic discrete facility location models
Model Description Assumptions Decision variables
Performance measures
Location set covering
model
Minimize total cost of providing
service
- Specify potential facility locations - Specify maximum acceptable distance (S) - Iterate on S
Location, number
Cost (dollars)
Maximal covering
location model
Maximize demand coverage
- Specify potential facility locations - Specify maximum acceptable distance (S) - Specify total number of facilities (P) - Iterate on S and P
Location, number
Demand coverage (percent)
P-median model
Minimize total weighted travel
distance
- Specify potential facility locations - Specify total number of facilities (P) - Iterate on P
Location Travel
distance (miles)
The key concept in covering models is the notion of coverage, with a maximum acceptable travel
distance specified such that a “demand node” (here, a patient’s 3-digit zip code home location)
can be served only by a facility within this distance. Covering models have been studied within
two major classes in the literature (Owen and Daskin, 1998). The first class is the location set
covering problem (LSCP), introduced by Toregas (1970), which aims to minimize the cost of
Chapter 4 - Network Optimization Models
90
facility location while ensuring that all demand nodes are covered. These models are useful for
cases in which all demand must be covered and one wishes to minimize the total cost to
accomplish this. The second class is the maximal covering location problem (MCLP), introduced
by Church and ReVelle (1974), which aims to maximize the level of coverage (i.e., the number
of covered demand nodes) using only a specified number of facilities and a maximum acceptable
travel distance. These models are useful in cases for which the cost of covering all demand nodes
is prohibitive. Finally, P-median models (ReVelle and Swain, 1970) seek to minimize the
average travel distance and are useful in cases for which a specific maximum acceptable distance
is less clear (Daskin and Dean, 2005). Many single and multiple objective extensions of these
three basic ideas have been developed over the past four decades to address particular problems
specific in manufacturing (Melachrinoudis and Min, 2000), distribution networks (Klose and
Drexl, 2005), web services (Aboolian et al., 2009), and other contexts. It also should be noted
that all of these types of models have continuous counterparts – for example planar LSCP and
MCLP (Church, 1984; Current and O'Kelly, 1992; Murray, 2001) and multi-source Weber
formulations (Cooper, 1963) – where facilities can be located anywhere in the plane.
4.2 Optimal Location and Use of In-Person and Video-Based PTSD Treatment Services
within the VHA
Given the vast medical and social costs of untreated or undertreated posttraumatic stress disorder
(PTSD), there is a critical need to improve access to evidence-based treatments (Murdoch et al.,
2005; Cohen et al., 2010; Ivanova et al., 2011). In the Veterans Affairs (VA), this need has
become of even greater importance with the development of effective treatments (Smith et al.,
2005; Davis et al., 2012; Schnurr and Lunney, 2012). However, access to these effective
Chapter 4 - Network Optimization Models
91
treatments, particularly for those residing in rural areas, is a practical issue. Innovative treatment
modalities, such as telehealth or video-based services, hold great promise to increase access to
these important mental health treatment services (Hailey et al., 2008; Wilson and Wells, 2009;
Mohr et al., 2010; Tutty et al., 2010), particularly in rural areas where access to in-person mental
health treatment is limited (Barnwell et al., 2012). Large integrated health systems such as VHA
can benefit from systems engineering tools to help plan the most appropriate ways to meet
mental health needs of patient populations. This study aims to apply longitudinal systems
engineering care location models to help determine the combined optimal geographic locations
and capacities for in-person care for veterans with PTSD across the New England VA network
and where instead video-based care would be advantageous over expensive within-system
capacity expansion or fee-based external care, if even feasible.
4.2.1 VA New England PTSD Treatment Services
The VA New England healthcare system is one of 21 Veterans Integrated Service Networks
(VISNs) within the Department of Veterans Affairs, with each VISN being regionally managed
and operating as a somewhat autonomous decision-making system. The VA New England VISN
(VISN 1) provides comprehensive medical services (including primary, mental health, specialty,
and hospital-based care) to more than 240,000 veterans residing in any of the six New England
states (Maine, New Hampshire, Vermont, Massachusetts, Rhode Island, and Connecticut)
annually. With an annual operating budget of approximately $2.2 billion, VISN 1 has 8 medical
centers (a total of 11 facilities as Connecticut and Boston healthcare system has 2 and 3
campuses, respectively), 40 community-based outpatient clinics, 5 outpatient clinics, 6 nursing
homes, and 2 domiciliaries – institutional houses for aged and disabled veterans (U.S.
Chapter 4 - Network Optimization Models
92
Department of Veterans Affairs, 2013). Nationwide, the VA spends more than $2.5 billion
annually to provide specialty care services, of which $125 million spent by VISN 1 (Veterans
Service Support Center, 2011a). While the VA has strived to develop integrated and coordinated
care, veterans who also are eligible for Medicare increasingly use non-VA specialty services,
resulting in increased fragmentation in care continuity (Liu et al., 2011). The fees for these out-
of-network services are reimbursed by the VA (“fee basis” visits) and can represent large
amounts of monies for some specialties. Better availability of specialty services within the VA
could decrease this fragmentation as well as result in lower overall costs.
Since the number of veterans diagnosed with PTSD is a function of access to care and may
under-represent true demand, overall PTSD prevalence among VA users rather than diagnostic
codes in VISN 1 was used to estimate the true need for PTSD care services. In November 2011,
the VA Office of Public Health reported that of the cumulative total of 741,954 Iraq and
Afghanistan veterans who utilized VA health services from October 2001 through September
2011, a total of 223,609 (30.14%) were diagnosed with PTSD (VA Office of Public Health,
2011a, 2011b). Given that the rate of PTSD diagnosis has been increasing over time (Rosenheck
and Fontana, 2007; Hermes et al., 2012; Shiner et al., 2012), a prevalence of 25% among
veterans who utilized VISN 1 VHA facilities from January 2009 through May 2010 was assumed
and true demand for PTSD care services over this time period was estimated based on this
prevalence. Figure 4-1 summarizes the estimated number of veterans with PTSD in VISN 1 by
state, with the majority residing in Massachusetts and Connecticut. Figure 4-2 depicts the
heterogeneous geographic distribution of veterans with PTSD in (a) raw counts and (b) per
square mile across VISN 1, with the locations of VA facilities.
Chapter 4 - Network Optimization Models
93
Figure 4-1 Estimated PTSD care demand by state across VISN 1 (January 2009 - May 2010)
Within the VA, veterans with PTSD are treated in specialized PTSD clinics, general mental
health clinics, or primary care clinics. Specialized PTSD clinics currently are located only at
some of the medical centers across New England. General mental health services are provided at
medical centers and large community-based outpatient clinics, while smaller community-based
outpatient clinics typically provide PTSD care either by referral to their affiliated medical center
or through the use of video-based telemental health services (Kussman, 2008). In VISN 1, six of
eight medical centers (Boston MA, Brocton MA, West Haven CT, Providence RI, Togus ME,
and White River Junction VT) have specialized PTSD care teams. Data between October 2009
and September 2010 shows that 35.15% of patients who received a primary diagnosis of PTSD
used specialized PTSD services across VISN 1, while more than half of those received care in
general mental health services (Veterans Service Support Center, 2011b), as presented in Figure
4-3.
11,521
8,992
5,738
3,956 3,721
2,540
0
2,000
4,000
6,000
8,000
10,000
12,000
MA CT ME NH RI VT
Dem
and
(num
ber o
f pat
ient
s)
State
Chapter 4 - Network Optimization Models
94
Figure 4-2 Geographic distribution of veterans with PTSD in (a) raw counts and (b) per square mile by 3-digit zip code and all VA facilities across VISN 1 (January 2009 - May 2010)
(a) (b)
Chapter 4 - Network Optimization Models
95
Figure 4-3 VA outpatient service use among patients with a primary PTSD diagnosis across VISN 1 (October 2009 - September 2010)
4.2.2 Methodology
4.2.2.1 Model Overview and Assumptions
The below model determines the optimal assignment of PTSD patients to the existing VISN 1
VHA facilities that minimizes overall total costs and calculates the total amount of mental health
manpower needed at each facility. A maximum acceptable travel distance (S) to a VA facility,
beyond which a patient is directed to an out-of-network provider at the VA’s expense, is
specified. Travel distances exceeding 30 miles are fully reimbursed by the VA at a current rate of
$0.70 per mile to cover gas, maintenance, vehicle depreciation, and inconvenience. Patients at
demand nodes (3-digit zip codes) assigned to an available VA facility within this acceptable
distance incur treatment and travel costs, while others are directed to an out-of-network provider
at a higher cost to the VA (i.e., non-coverage cost). Given that most private practice providers
lack knowledge of PTSD and other mental health disorders prevalent among veterans and that
they are unfamiliar with the VA’s treatment resources for such conditions (Boscarino et al.,
2010; Kizer, 2012), demand nodes that are sent to a non-VA provider are considered candidates
Mental health services, 56.35%
(11,606 patients) Specialized PTSD
services, 35.15%
(7,239 patients)
Other services,
8.51% (1,752 patients)
Chapter 4 - Network Optimization Models
96
for video-based services as a more effective alternative both in terms of quality and cost of care.
Other model assumptions include
(1) Each patient receives care at the facility to which he is assigned,
(2) Demand is deterministic, equally distributed over time, and with no seasonality effects,
(3) Care cost variations between patients, medical centers, and geographic regions are
negligible,
(4) PTSD treatment preferences among VA users are assumed to be a) 60% want evidence-
based psychotherapy only, b) 25% want medication only, and c) 15% want both (Watts et
al., 2012),
(5) Assumptions on evidence-based psychotherapy included: a) Conducted by
psychotherapists (with at minimum a master’s degree), b) consisted of one evaluation, 12
treatment sessions, and three phone calls, and c) a total of 13.75 clinical hours needed per
patient per year (i.e., one hour for evaluation, one hour for each treatment session, and 15
minutes for each phone call) (Kussman, 2008; U.S. Department of Veterans Affairs and
Department of Defense, 2010),
(6) Medication assumptions included: a) Prescribed by psychiatrists, b) consisted of one
evaluation session, six follow-up visits, and three phone calls, and c) a total of 4.75
clinical hours needed per patient per year (i.e., one hour for evaluation session, 30
minutes for each follow-up visit, and 15 minutes for each phone call),
Chapter 4 - Network Optimization Models
97
(7) Mental health providers located in specialized PTSD clinics only focus on patients with
PTSD, while those in general mental health and primary care clinics spend 75% of their
time addressing disorders other than PTSD.
4.2.2.2 Integer Programming Model
The following notation and input parameters are used in the mathematical model, with
i = index of geographic demand nodes,
j = index of facilities,
hiP = total number of patients who need psychotherapy at demand node i,
hiM = total number of patients who need medication at demand node i,
vP = total number of psychotherapy visits needed per year per patient,
vM = total number of medication visits needed per year per patient,
kP = total clinical hours needed for psychotherapy per year per patient,
kM = total clinical hours needed for medication per year per patient,
S = maximum acceptable travel distance,
tij = distance between demand node i and facility j,
C1P = total cost of psychotherapy per year per patient,
C1M = total cost of medication per year per patient,
C2 = travel cost per mile,
C3 = non-coverage cost per year per patient,
njP = total number of psychotherapists needed at facility j per year,
njM = total number of psychiatrists needed at facility j per year,
T = total working hours of a therapist/psychiatrist per year,
O = set of medical centers that have specialized PTSD clinics,
lj = rate of time that a therapist/psychiatrist spends for treating patients with PTSD in facility j, equal to 1 if j ∈ O, and 0.25 otherwise,
D = total demand (for both psychotherapy and medication) in the corresponding planning horizon, and
Chapter 4 - Network Optimization Models
98
dij = reimbursed travel distance, equal to tij if tij > 30, and 0 otherwise.
The decision variable is
Yij = 1 if patients in demand node i are treated at facility j, and 0 otherwise.
Using the above notation, an integer programming model can be written as
Minimize �∑ ∑ Yijji �hiP�C1
P + vPdijC2� + hiM�C1
M + vMdijC2���
+C3 �D− ∑ ∑ Yijji �hiP + hi
M��
Subject to
∑ Yijj ≤ 1 ∀ i (4-1)
∑ Yijj|tij>S ≤ 0 ∀ i (4-2)
njP = ∑ Yijhi
PkP �ljT��i ∀ j (4-3)
njM = ∑ Yijhi
MkM �ljT��i ∀ j (4-4)
Yij ∈ {0, 1} ∀ i, j (4-5)
The objective function minimizes the total of treatment, travel and non-coverage costs. In the
first term, treatment (i.e., psychotherapy and medication) and travel costs are calculated for the
patients who receive care within-network. The second term represents the non-coverage cost
calculated for the remaining patients that are not covered by any VA facility. Constraint (4-1)
assigns every demand node to at most one VA facility. Constraint (4-2) ensures that a demand
node can be assigned to a facility if and only if the travel distance is less than or equal to the
acceptable maximum. The total number of psychotherapists and psychiatrists needed in each
Chapter 4 - Network Optimization Models
99
facility is calculated by constraints (4-3) and (4-4). Constraint (4-5) defines the decision variable
Yij to be binary.
4.2.3 Application and Results
4.2.3.1 Application Details
The mathematical model was used to determine for each facility the optimal amount of each type
of PTSD care service to provide and for each PTSD patient whether service could be
economically provided within a specified travel distance to any existing PTSD treatment facility.
Assignments of care type to facilities were made regardless of current services at them, with
assignment of PTSD patients to sites used to determine which patients ideally should be seen
where. VA goals for maximum acceptable travel distances (currently 30 miles for PTSD care)
were used to determine these service locations, patient destinations, and if a particular patient
could receive within-network in-person care. Those that could not receive in-person care within
the VISN 1 network were considered candidates for video-based care rather than outside the VA
system in a manner so as to minimize total care and delivery costs. This analysis was also
repeated for other maximum travel distances in order to investigate how results might be
impacted by this somewhat arbitrary service threshold.
4.2.3.2 Results
Figure 4-4 shows the tradeoff between maximum acceptable driving distance, total minimal cost,
and access for the estimated 2010 PTSD care demand. Also shown is the current situation for
comparison, estimated based on the current patient allocations, which has an 11.08% ($10.9
million) higher cost and 56.26% lower access than even the optimized case with no telehealth
Chapter 4 - Network Optimization Models
100
service due to current sub-optimal patient-to-facility allocation. Utilizing telehealth services
where demand cannot be covered by the VA within 30 miles, rather than providing care at non-
VA facilities, would result in further savings of $4,001,664 (4.04%) given the same access
coverage, or alternately a roughly 50% reduction in maximum driving distance to 30 miles given
the same cost. Note that all demand can be covered by the VA if a maximum acceptable driving
distance of more than 60 miles is allowed. This, however, causes a 45% increase in average
driving distance per patient that exceeds the VA’s maximum travel distance policy.
Figure 4-4 Tradeoff between acceptable distance and (a) total annual cost, and (b) coverage percentage assuming estimated care demand in 2010
$80
$90
$100
$110
$120
$130
$140
10 20 30 40 50 60 70 80 90 100
Annu
al co
st (i
n m
illio
ns)
Maximum acceptable travel distance (S) (miles)
(a) Total annual cost
Current cost
No telehealth
With telehealth
20%
30%
40%
50%
60%
70%
80%
90%
100%
10 20 30 40 50 60 70 80 90 100
Cove
rage
per
cent
age
Maximum acceptable travel distance (S) (miles)
(b) Coverage percentage
Current
No telehealth
With telehealth
Chapter 4 - Network Optimization Models
101
Table 4-2 summarizes the resulting optimal locations of in-person PTSD services and the
number of psychotherapist and psychiatrist FTEs required at each site to provide this care. As
shown, rural pockets of Maine, Vermont, and New Hampshire would be better served with
video-based mental health services since demand in these areas cannot be covered by any of the
VA facilities. In Massachusetts, Connecticut, and Rhode Island, there is sufficient need and
access for in-person PTSD services, with the possible exception of the medical center located in
Jamaica Plain, MA which might be treated in the next closest location instead since only an
estimated 17 patients here will need care during the year.
Figure 4-5 illustrates how optimal within-network and video-based care locations change if some
other maximum travel distances were allowed. Finally, Figure 4-6 summarizes future projected
needs for PTSD treatment by therapists and psychiatrists, along with the estimated number of
veterans needing PTSD care. Future PTSD care demand was estimated via the VA’s veteran
population projections (www.va.gov/VETDATA/Demographics/Demographics.asp) and
geographically distributed among all 59 VISN 1 zip codes based on current relative proportions.
As expected, the number of providers required decreases over the next 10 years as a result of
shrinking veteran population projections, while zip codes that would be better served with video-
based services remain the same.
Chapter 4 - Network Optimization Models
102
Table 4-2 Optimal allocation of patient zip codes to VISN 1 facilities for a 30-mile maximum acceptable distance assuming estimated care demand in 2010
VISN 1 VA facilities Type of care provided
Patient 3-digit zip codes
Total number of patients assigned
Total number of psychotherapists needed (FTEs)
Total number of psychiatrists
needed (FTEs)
Brocton, MA PCT 023, 027 1687 16.12 2.61
Jamaica Plain, MA PCT 022 17 0.16 0.03
West Roxbury, MA MHS 020, 024 801 30.61 4.95
Newington, CT MHS 060, 061, 064 4570 174.71 28.24
West Haven, CT PCT 065, 066 1336 12.76 2.06
Manchester, NH MHS 030, 031 1257 48.04 7.76
Providence, RI PCT 028, 029 3722 35.57 5.75
Togus, ME PCT 043 598 5.71 0.92
White River Junction, VT PCT 037, 050 711 6.79 1.10
Fitchburg, MA MHS 014 471 17.99 2.91
Gloucester, MA Rf/TH 019 832 31.80 5.14
Dorchester, MA Rf/TH 021 1769 67.62 10.93
Framingham, MA MHS 017 353 13.47 2.18
Lowell CBOC, MA MHS 018 1157 44.22 7.15
Worcester, MA MHS 015, 016 1085 41.47 6.70
New London, CT MHS 063 973 37.19 6.01
Stamford, CT MHS 068, 069 737 28.17 4.55
Waterbury, CT MHS 067 948 36.24 5.86
Windham, CT MHS 062 430 16.42 2.65
Somersworth CBOC, NH MHS 038, 039 1037 39.65 6.41
Tilton CBOC, NH MHS 032, 033 737 28.17 4.55
Greenfield, MA MHS 013 275 10.51 1.70
Pittsfield, MA Rf/TH 012 181 6.91 1.12
Springfield, MA MHS 010, 011 1974 75.44 12.19
Hyannis, MA MHS 026 657 25.09 4.05
Martha’s Vineyard, MA Rf/TH 025 267 10.19 1.65
Bangor CBOC, ME MHS 044 822 31.43 5.08
Portland CBOC, ME Rf/TH 040, 041 1052 40.22 6.50
Rumford CBOC, ME MHS 042 963 36.80 5.95
Bennington CBOC, VT MHS 052 176 6.73 1.09
Brattleboro CBOC, VT Rf/TH 053 137 5.24 0.85
Colchester CBOC, VT MHS 054 771 29.47 4.76
Newport CBOC, VT Rf/TH 058 320 12.22 1.98
Rutland CBOC, VT Rf/TH 057 321 12.25 1.98
Non-covered (candidates for video services) 034, 035, 036 (NH); 045, 046, 047, 048, 049 (ME); 051, 056, 059 (VT)
PCT: Specialized PTSD care teams, MHS: General mental health services, Rf/TH: By referral or telemental health
Chapter 4 - Network Optimization Models
103
Figure 4-5 Optimal areas for video-based services for a (a) 15-mile, (b) 30-mile, (c) 45-mile, and (d) 60-mile maximum acceptable distance assuming estimated care demand in 2010
(a) (b)
(c) (d)
Chapter 4 - Network Optimization Models
104
Figure 4-6 Estimates of future PTSD care needs over the next 10 years (2010 - 2020)
4.2.4 Discussion
This study highlights the need for appropriate allocation of PTSD services for veterans. Results
may be useful as a planning tool for PTSD resource allocation within the U.S. Department of
Veteran Affairs. Access to care is a particular concern for veterans who are unable to readily
access PTSD treatment, such as those living in rural environments. In such cases, new treatment
modalities such as telehealth (i.e., via video conference) can be considered. Such treatment
actually may be preferred by some veterans given that it increases convenience and reduces
privacy concerns.
For the VA population, most clinic sites have sufficient volume to justify in-person staff for face-
to-face PTSD treatment. There are a small number of community-based clinics, however, whose
estimated demand is insufficient to justify in-person staff. For these sites, this analysis suggests
that PTSD treatment needs would be better met by video-based services. In general, these sites
were in rural or highly rural regions of New England, primarily in central Vermont, southwestern
1,035 1,004 972 942 912 884 857 830 805 781 757
167 162 157 152 147 143 138 134 130 126 122
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
0
200
400
600
800
1,000
1,200
1,400
2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020
Num
ber o
f pat
ient
s
Num
ber o
f pro
vide
rs (F
TEs)
nee
ded
Year
Number of therapistsNumber of psychiatrists
Chapter 4 - Network Optimization Models
105
New Hampshire, and much of northern Maine. All other areas have sufficient need to support in-
person services, assuming availability.
Study limitations include logic, data, and preference assumptions in the mathematical model. For
example, it is assumed that providers trained in treating PTSD are available or could be relocated
to each clinic site, which may not be possible immediately, although in such cases additional or
temporary reliance on telemental health services may be possible. Assumptions regarding patient
treatment preferences also were based on a pilot sample (Watts et al., 2012) and were assumed to
be equally distributed across the patient population. Rural patients instead, for example, may
have higher preference for medication treatment. It also is not clear that all VA users have a
preference for or an understanding of evidence-based psychotherapy. While the VA follows
guidelines that promote evidence-based treatment (Kussman, 2008; U.S. Department of Veterans
Affairs and Department of Defense, 2010; Cloitre et al., 2012), furthermore, the definition of
evidence-based psychotherapy is not universal and instead could be defined as any of those listed
in the National Registry of Evidence-based Programs and Practices (Miller et al., 2006;
Sorensen, 2011). In addition to psychiatrists, medications also can be prescribed by nurse
practitioners, physician assistants, or primary care physicians and therefore the estimate for
medications prescribed may be conservative. The recent RESPECT-PTSD trial (Schnurr et al.,
2013) also showed the difficulty of implementing evidence-based care for PTSD by primary care
providers. Finally, results have based only on the New England VA network, although the
general value of this approach seems generalizable to all VISNs.
Additional research could examine if tele treatment modalities affect treatment retention among
veterans or the active duty population. More generally, use of operations research and systems
Chapter 4 - Network Optimization Models
106
engineering methods appears useful to help inform these types of macro system design and
policy issues. Similar analysis for the location of other evidence-based mental health services
therefore also may be beneficial.
4.3 Single and Multi-Period Location-Allocation Models for Sleep Apnea Testing Services
within the VHA
Along with the care services for combat-related conditions such as TBI and PTSD, VA also is
responsible for providing comprehensive medical services to the veterans with other chronic
health problems. There has been, for instance, an increased demand for sleep apnea testing and
treatment services among Vietnam veterans in their 50s and 60s given the high prevalence of
sleep disorders in this age group (Ancoli-Isreal et al., 1991; Young et al., 1993; Punjabi et al.,
2000; Rosenheck and Fontana, 2007). In the New England VA network (VISN 1), it is reported
that there is insufficient access to sleep apnea testing services, causing a significant increase in
the number of “fee basis” visits and thus cost. This study aims to illustrate the use of
mathematical optimization models to help improve the geographic locations and capacities of
sleep apnea testing services across the New England VA network in a manner so as to provide
appropriate access at minimal cost considering patient needs and preferences.
4.3.1 VA New England Sleep Apnea Services
Sleep apnea is a disorder characterized by abnormal pauses in breathing or instances of
abnormally low breathing during sleep and has been linked with high blood pressure, heart
problems, stroke, fatigue, headaches, snoring, daytime drowsiness, and memory problems
(Ancoli-Isreal and Avalon, 2006; Erman, 2006). These health problems occur at a much higher
Chapter 4 - Network Optimization Models
107
rate among veterans than in the general population due to long-term exposure to dust, smoke,
chemicals, and other environments (U.S. Department of Health and Human Services, 2012). Up
to 20% of all U.S. veterans currently suffer from sleep apnea, with an annual treatment cost of
$225 million to the VA (Veterans Service Support Center, 2011a).
Figure 4-7 summarizes the number of treated sleep apnea patients in VISN 1 by state, with the
majority residing in Connecticut and Massachusetts as expected given greater overall
populations. Since observed utilization is a function of capacity and may under- or over-
represent true demand, also shown is an estimate of the greater true need over this time period,
based on clinical input and patient-specific predictors, such as their body mass index (since it is
well-known that the risk for sleep apnea is higher for people who are overweight (U.S.
Department of Health and Human Services, 2012)). Figure 4-8 shows estimated demand by 3-
digit zip code relative to VA facilities across VISN 1.
Figure 4-7 Sleep apnea care demand by state across VISN 1 (January 2009 - May 2010)
1,774
1,326
566 455 455
136
2,502
3,178
1,120 1,023
1,640
723
0
400
800
1,200
1,600
2,000
2,400
2,800
3,200
CT MA NH RI ME VT
Dem
and
(num
ber o
f vis
its)
State
Observed utilizationEstimated true need
Chapter 4 - Network Optimization Models
108
Figure 4-8 Estimated sleep apnea care demand by 3-digit zip code and all VA facilities across VISN 1 (January 2009 - May 2010)
A definitive diagnosis of sleep apnea is made through use of overnight polysomnographic testing
that involves having a patient sleep in a polysomnogram bed (referred to as a “sleep bed”) with
continuous monitoring of their blood oxygen levels, electroencephalogram, and vital signs. A
general belief has been that inadequate access to overnight polysomnographic testing in VISN 1
is causing a large number of veterans to be referred to out-of-network providers. The fees for
Chapter 4 - Network Optimization Models
109
these out-of-network services are reimbursed by the VA (“fee basis” visits) and can represent
large amounts of monies for some specialties. In the sleep apnea case, data between January
2009 and May 2010 shows that of 4712 tested patients from VISN 1, 3927 were tested in one of
the VA facilities at a total operation and travel cost of $2,431,762, with 785 tested outside the
network at a total cost of $667,250 or 9% of the overall cost.
Currently, only four of eight VA medical centers in VISN 1 (i.e., West Haven CT; West Roxbury
MA; Manchester NH; and Providence RI) have sleep laboratories that provide polysomnographic
testing. Closing these laboratories or decreasing their capacities is not feasible within the VA
given policy and other considerations. Table 4-3 summarizes the number of sleep beds and visits
at each facility.
Table 4-3 Number of sleep beds, within-network and outside-network sleep apnea patients in VISN 1 (January 2009 - May 2010)
Facilities Current number of sleep beds
Number of within-network visits
Number of outside-network visits
West Haven, CT 4 1,738 255 West Roxbury, MA 6 1,119 - Manchester, NH 4 476 - Providence, RI 2 434 - Togus, ME* - 97* 286 White River Junction (WRJ), VT - - 204 Total 16 3,767 745
*In 2009, there were two sleep beds in Togus.
The West Haven medical center generated outside-network fee visits over this time period, even
though it has sleep beds, due to demand at times exceeding capacity as shown in Figure 4-9.
However, a relatively large number of fee type visits occurred in months in which utilization
(measured as the percent of capacity usage) was low. Fee type utilization associated with other
Chapter 4 - Network Optimization Models
110
facilities with sleep beds suggests regional demand does not exceed regional capacities, as shown
in Figure 4-10. Due to the fact that patient records are based on billing rather than visit dates,
utilizations in some months are over 100%, which may not reflect actual utilization levels. Those
observations underscore data inaccuracies and idiosyncrasies, the need for sensitivity analyses,
and the value of working in an integrated analyst-clinician research team.
Figure 4-9 Fee visits for West Haven, (a) number of fee visits versus monthly utilization, (b) weekly fee visits versus in-house visits (months with relatively low utilization (< 50%) and high fee visits are circled)
6
12
22
16
12 13 13 13 16
32
12 10
13
9
20
16
20
0%
20%
40%
60%
80%
100%
120%
140%
160%
0
4
8
12
16
20
24
28
32
Util
izat
ion
Num
ber o
f fee
-typ
e vi
sits
Month
(a)
Number of fee-type visitsUtilization
59
24 24
13 18
34 31
26 22 24 23 21 20
63
39
31 36
25 23
50 45
5 3 2 2 2 2 3 2 3 4 3 5 6 5 6 3 2
8 4 4 3
0
10
20
30
40
50
60
70
Num
ber o
f vis
its
Week
(b)
House-type Fee-type
Chapter 4 - Network Optimization Models
111
Figure 4-10 Utilization of sleep apnea capacity by medical center over time (utilizations over 100% are circled)
Figure 4-11 summarizes the distances patients travelled to receive within-network care over the
same time period, with a mean of 44.5 miles but a high variability ranging from 2 to 420 miles.
Approximately 70% of patients travelled more than 30 miles and 25% more than 60 miles,
typical acceptable distance thresholds. Roughly 5% of patients traveled more than 100 miles. As
mentioned earlier, travel distances exceeding 30 miles are fully reimbursed by the VA at a
current rate of $0.70 per mile.
Figure 4-11 Travel distance distribution, VISN 1 (January 2009 - May 2010)
0%
20%
40%
60%
80%
100%
120%
140%
160%
With
in-n
etw
ork
capa
city
util
izat
ion
Month
West HavenWest RoxburyManchesterProvidenceTogus
100%
31.58%
44.80%
17.74%
3.44% 2.44%
0%
10%
20%
30%
40%
50%
0-30 31-60 61-90 91-120 >120
Perc
ent o
f pat
ient
s
Travel distance (miles)
Chapter 4 - Network Optimization Models
112
4.3.2 Methodology
4.3.2.1 Model Overview and Assumptions
The below models determine the optimal sleep bed capacity in each existing VISN 1 VHA
facility that minimizes overall total costs over a single or multiple periods, relative to geographic
demand. The single-period model is useful for short term planning or for cases in which future
demand is unlikely to change, whereas the multi-period model is important in cases for which
demand or patient volume may significantly change over time, as can be the case with veteran
populations.
Similar to the above model developed for the case of PTSD treatment services, a maximum
acceptable travel distance (S) to a VA facility is specified, beyond which a patient is directed to
an out-of-network provider at the VA’s expense. Patients at demand nodes (3-digit zip codes)
assigned to an available VA facility within this acceptable distance are referred to as in-house
type visits and incur procedure costs as well as travel costs, as described above. Other model
assumptions include
(1) Each patient receives his care at the facility to which he is assigned and spends one night
there,
(2) Annual capacity of a sleep bed is equal to the number of working days in a year,
(3) Facilities that currently provide this care continue to do so and capacity down-sizing is
not allowed,
Chapter 4 - Network Optimization Models
113
(4) Procedure costs include overhead and labor costs independent from a patient’s health
state,
(5) Demand is deterministic, equally distributed over time, and with no seasonality effects,
and
(6) Care cost variations between medical centers and geographic regions are negligible.
The first four assumptions reflect the current circumstances of the VA sleep apnea testing
services. The fifth and sixth assumptions also are realistic since seasonality is not indicated in the
literature as a major factor in sleep apnea cases and there is not a significant cost variation
between VISN 1 facilities all of which operate under the same local administration. These two
assumptions, in fact, decrease the complexity of the model.
The multi-period model considers an n-year planning horizon, with a user-specified maximum
number of facilities P that can be added each year (assumed the same for each period). An
underutilization cost is included in the multi-period model since expansion decisions may
produce over capacity in some following years. Facilities in which service is added or expanded
also are assumed to continue to operate at this capacity level over the remainder of the planning
horizon.
4.3.2.2 Single-Period Model
The following notation and input parameters are used in the single-period model, with
i = index of geographic demand nodes,
Chapter 4 - Network Optimization Models
114
j = index of facilities, hi = total number of sleep apnea patients at demand node i, P = maximum number of additional facilities to be located,
S = maximum acceptable travel distance,
C1 = in-house procedure cost per patient,
C2 = travel cost per mile,
C3 = non-coverage cost per patient,
C4 = cost of adding a sleep bed,
Kj = maximum capacity of facility j, tij = distance between demand node i and facility j, fj = initial number of beds in facility j, T = total number of days in the given period,
O = set of facilities that provide service currently,
D = total demand in the corresponding planning horizon, and
dij = reimbursed travel distance, equal to tij if tij > 30, and 0 otherwise.
The decision variables are
Xj = 1 if facility j provides sleep apnea service and 0 otherwise,
Yij = 1 if patients in demand node i are treated at facility j and 0 otherwise, and
aj = number of additional sleep beds to add in facility j.
Using the above notation, an integer programming mathematical model can be written as
Minimize �∑ ∑ Yijhiji �C1 + dijC2��+ C3�D− ∑ ∑ Yijhiji � + C4 ∑ ajj
Subject to
∑ Xjj∉O ≤ P (4-6)
∑ Yijj ≤ 1 ∀ i (4-7)
Yij − Xj ≤ 0 ∀ i, j (4-8)
Chapter 4 - Network Optimization Models
115
∑ Yijj|tij>S ≤ 0 ∀ i (4-9)
∑ Yijhii ≤ T �fj + aj� ∀ j (4-10)
∑ Yijhii ≤ Kj ∀ j (4-11)
Xj = 1 ∀ j ∈ O (4-12)
Xj ∈ {0, 1} ∀ j ∉ O (4-13)
Yij ∈ {0, 1} ∀ i, j (4-14)
aj integer ∀ j (4-15)
The objective function minimizes the total of in-house, fee, and set-up cost for additional beds.
In the first term of the objective function, treatment and travel costs are calculated for the
patients who receive in-house type service. The second term represents the fee type service cost
calculated for the remaining patients that are not covered by any VA facility. The weighted sum
of non-covered demand nodes is multiplied by the fixed external visit cost to calculate the total
cost of fee type visits. It should be noted that the term “C3D” in the fee type cost equation could
be excluded from the objective function since it is constant and non-optimizable, but is included
here for completeness. Finally, the last term multiplies the total number of additional sleep beds
in all facilities by a fixed bed set-up cost. Constraint (4-6) requires that service is added to at
most P additional facilities, while constraint (4-7) assigns every demand node to at most one VA
facility (those unassigned receive their care outside the network). Constraints (4-8) and (4-9)
ensure that a demand node can be assigned to a facility if and only if this facility provides this
type of care service and the travel distance is less than or equal to the acceptable maximum,
respectively. The number of additional beds at each facility is determined by constraint (4-10).
Constraint (4-11) ensures that the maximum capacity of each facility is not exceeded. Constraint
(4-12) ensures that all facilities currently providing service remain open. Constraints (4-13)
Chapter 4 - Network Optimization Models
116
through (4-15) define the location (Xj), allocation (Yij), and capacity expansion (aj) decision
variables to be binary or non-negative integers.
4.3.2.3 Multi-Period Model
The multi-period model expands the above to include the additional notation and parameters,
with most inputs and decision variables now having a time period index
t = index of period,
hit = total number of patients at demand node i during period t,
C5 = unused sleep bed cost per day, and
Dt = total demand during period t,
with the decision variables now being
Xjt = 1 if facility j provides sleep apnea service during period t, and 0 otherwise,
Yijt = 1 if demand node i is covered by facility j during period t, and 0 otherwise,
ajt = number of additional sleep beds in facility j during period t, and
bjt = number of total sleep beds in facility j at the beginning of period t (an auxiliary decision variable).
The mathematical model then extends to
Minimize �∑ ∑ ∑ Yijthi
tjit �C1 + dijC2��+ C3 ∑ (Dt –∑ ∑ Yij
thit
ji )t +C4 ∑ ∑ ajjt + C5 ∑ ∑ �T�bj
t + ajt� − ∑ Yij
thit
i �jt
Subject to
∑ Xjt
j∉O ≤ P ∀ i (4-16)
Xjt ≥ Xj
t−1 ∀ j, t ∈ {2, 3, 4, 5, 6} (4-17)
Chapter 4 - Network Optimization Models
117
bj1 = fj ∀ j (4-18)
bjt = bj
t−1 + ajt−1 ∀ j, t ∈ {2, 3, 4, 5, 6} (4-19)
∑ Yijthi
ti ≤ T�bj
t + ajt� ∀ j, t (4-20)
∑ Yijt
j ≤ 1 ∀ i, t (4-21)
Yijt − Xj
t ≤ 0 ∀ i, j, t (4-22)
∑ Yijt
j|tij>S ≤ 0 ∀ i, t (4-23)
∑ Yijthi
ti ≤ Kj ∀ j, t (4-24)
Xjt = 1 ∀ j ∈ O, t (4-25)
Xjt ∈ {0, 1} ∀ j ∉ O, t (4-26)
Yijt ∈ {0, 1} ∀ i, j, t (4-27)
ajt integer ∀ j, t (4-28)
The objective function now includes a fourth term that represents the total underutilization cost,
where the number of empty beds at each facility is the difference between its total capacity and
total allocated patients. Constraint (4-16) ensures that at most P number of sleep laboratories can
be located in each period. Closing a service over the remaining planning horizon per VISN 1
policy is prevented by constraint (4-17), although this could be considered in future models.
Constraints (4-18) and (4-19) calculate the number of beds in each facility at the beginning of
each period, and constraint (4-20) determines the number of needed beds in each facility at each
period. Constraints (4-21)-(4-28) function much as their counterparts in the single-period model
(constraints (4-7)-(4-9) and (4-11)-(4-15)).
Chapter 4 - Network Optimization Models
118
4.3.3 Application and Results
4.3.3.1 Application Details
The above models were solved using both the observed and predicted geographic demand
patterns described earlier. Future demand for the next five years was estimated via the VA’s
veteran population projections, summarized in Figure 4-12, with significant estimated decreases
over the next 20 years due to aging veteran populations and smaller modern military sizes. The
implication of these trends is that in the short term it may be optimal to source a large percent of
care outside the network so as to not overbuild long-term excess capacity. These future annual
demand estimates were geographically distributed across the 59 zip code demand nodes based on
the current relative proportions.
Figure 4-12 Total projected sleep apnea demand by year for entire VISN 1 (New England)
Based on discussions with VISN 1 leadership, for both types of models three sets of facilities
were considered for expansion:
10,187
9,878
9,574
9,277
8,991
8,714
7,500
8,000
8,500
9,000
9,500
10,000
10,500
2010 2011 2012 2013 2014 2015
Tota
l est
imat
ed d
eman
d (n
umbe
r of p
atie
nts)
Year
Chapter 4 - Network Optimization Models
119
− Case 1: All seven existing VA medical centers in New England,
− Case 2: All seven existing VA medical centers and the five largest outpatient clinics (Burlington VT, Springfield MA, Worcester MA, Portland ME, Bangor ME),
− Case 3: All seven existing VA medical centers and all 45 outpatient clinics across New England, regardless of size.
Cost parameters provided by VISN 1 are summarized in Table 4-4.
Table 4-4 Cost parameters used in mathematical models
Notation Explanation Cost ($)
C1 In-house procedure cost per patient $ 588 C2 Travel cost per mile $ 0.70 C3 Non-coverage cost per patient $ 850 C4 Cost of adding a sleep bed (capacity expansion) $ 21,000 C5 Unused sleep bed (underutilization) cost per day $ 385
The mathematical models were solved using the LINGO 11 software on a Dell desktop computer
with 3 GB of RAM running a 1.8 GHz Intel Core 2 Duo processor. Both models were run for
almost 100 scenarios in order to examine different coverage distance policies (S), numbers of
allowable additional facilities (P), and candidate locations. With larger coverage distances or
candidate locations, the feasible region and thus run times significantly increase (up to 44
minutes), as illustrated in Table 4-5 for the multi-period model.
Chapter 4 - Network Optimization Models
120
Table 4-5 Multi-period model solution times based on different scenarios for case 1, 2, and 3
Case Acceptable travel distance (S) (miles)
Solution times (in seconds) P = 1 P = 2 P = 3
1 (7 candidate locations)
20 <1 <1 <1 30 <1 1 <1 40 1 2 3 50 2 10 11 60 12 16 16
2 (12 candidate
locations)
20 1 <1 1 30 3 1 2 40 3 2 3 50 2 2 3 60 34 154 45
3 (52 candidate
locations)
20 2 1 1 30 6 5 9 40 10 13 42 50 9 14 29 60 71 298 312
4.3.3.2 Single-Period Results
For the single-period model, Figure 4-13 summarizes the optimized tradeoff between maximum
acceptable driving distance, total minimal cost, and access assuming current observed demands
and that sleep apnea care can be added at zero through five additional facilities. Also shown is
the current situation for comparison, which has a 2.3% higher cost and 4.5% lower access than
even the “zero” case with no added capacity nor facilities, due to sub-optimal patient-to-facility
allocation. Increasing the number of new facilities to three (P = 3) would result in a savings of
$145,188 (4.7%) given the same access coverage, a roughly 50% reduction in maximum driving
distance to 50 miles given the same cost, or any pairwise cost-distance point on the shown curve.
After three new facilities, the cost-access tradeoff starts to become negligible, indicating
expansion beyond P > 3 facilities may not be useful for practical purposes.
Chapter 4 - Network Optimization Models
121
Figure 4-13 Optimized tradeoff between acceptable distance and (a) total cost, and (b) coverage percentage for single-period model assuming current observed demands and case 3
Table 4-6 summarizes general results and savings for the single-period model assuming
estimated true demands and based on a 60-mile maximum travel distance, showing the costs of
fee and in-house visits. The rows with P = 0 correspond to the cases of modified capacities in
existing facilities but not additional sleep care anywhere else.
$2,900
$3,050
$3,200
$3,350
$3,500
$3,650
$3,800
20 30 40 50 60 70 80 90 100
Cost
(in
thou
sand
s)
Maximum accetable travel distance (S) (miles)
(a) Total cost
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
20 30 40 50 60 70 80 90 100
Cove
rage
Maximum acceptable travel distance (S) (miles)
(b) Coverage percentage
P=0 P=1 P=2 P=3 P=4 P=5
Current cost
Current approximate 95th travel distance
percentile
Current
Current approximate 95th travel distance
percentile
P=0 P=1 P=2 P=3 P=4 P=5
Chapter 4 - Network Optimization Models
122
Table 4-6 Results for single-period model with 60-mile maximum acceptable travel distance
Case Num. add. facilities
(P) Total cost
In-house type visit
cost
Fee type visit cost
% in-house type
visits
Optimal facility locations (total number of beds)** Savings
1 (7
candidate locations)
Est. obs.* $7,455,189 $3,252,421 $4,202,768 51% - -
0 $7,274,793 $3,494,030 $3,759,762 53% - $110,085
1 $7,020,768 $4,342,393 $2,573,375 70% Northampton(4) $434,421
2 $6,874,504 $4,896,599 $1,809,905 79% Togus(3)-Northampton(4) $580,685
3 $6,810,171 $5,102,943 $1,518,228 82% Togus(3)-Northampton(4)-WRJ(1) $645,018
2 (12
candidate locations)
Est. obs.* $7,455,189 $3,252,421 $4,202,768 51% - -
0 $7,274,793 $3,494,030 $3,759,762 53% - $110,085
1 $7,007,062 $4,296,089 $2,605,972 70% Springfield(4) $448,127
2 $6,860,798 $4,850,296 $1,842,502 79% Togus(3)-Springfield(4) $594,391
3 $6,775,736 $5,134,086 $1,452,650 83% Togus(3)-Springfield(4)-Portland(2) $679,453
3 (52
candidate locations)
Est. obs.* $7,455,189 $3,252,421 $4,202,768 51% - -
0 $7,274,793 $3,494,030 $3,759,762 53% - $110,085
1 $7,007,062 $4,296,089 $2,605,972 70% Springfield(4) $448,127
2 $6,860,798 $4,850,296 $1,842,502 79% Togus(3)-Springfield(4) $594,391
3 $6,774,937 $5,153,385 $1,411,553 84% Togus(3)-Springfield(4)-Saco(2) $680,252 *Estimated observed case **For all cases, optimum solution also requires a capacity expansion of one bed at the current facility in Providence.
Figure 4-14 illustrates similar information and conclusions now using the estimated true
demands. Table 4-7 further summarizes optimal locations and their corresponding capacities,
using true demand, for 27 different scenarios based on coverage policy, additional number of
facilities, and set of candidates. Patterns in this table provided important insight for decision
makers. For example, capacity expansion at the current sleep laboratory in Providence always is
suggested in all scenarios, removing most debate about the accuracy of various assumptions and
demand estimates. Several other candidates also are frequently suggested as optimal expansion
facilities, namely, Newington, Togus, Northampton, and Springfield.
Chapter 4 - Network Optimization Models
123
Figure 4-14 Optimized tradeoff between acceptable distance and (a) total cost, and (b) coverage percentage for single-period model assuming estimated true demand and candidate facilities under case 3
$6,500
$6,750
$7,000
$7,250
$7,500
$7,750
$8,000
$8,250
20 30 40 50 60 70 80 90 100
Cost
(in
thou
sand
s)
Maximum acceptable travel distance (S) (miles)
(a) Total cost
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
20 30 40 50 60 70 80 90 100
Cove
rage
Maximum acceptable travel distance (S) (miles)
(b) Coverage percentage
P=0 P=1 P=2 P=3 P=4
Current
Current approximate 95th travel distance
percentile
Current cost
Current approximate 95th travel distance
percentile
P=0 P=1 P=2 P=3 P=4
Chapter 4 - Network Optimization Models
124
Table 4-7 Optimal facility locations for single-period model based on estimated true demand (numbers in parentheses represent the number of additional beds in corresponding facilities)
S (miles) P
Case 1 Case 2 Case 3
Facility Beds Capacity expansion Facility Beds Capacity
expansion Facility Beds Capacity expansion
30
1 Newington 4 Providence (+2) Newington 4 Providence (+2) Newington 4 Providence (+2)
2 Newington Northampton
4 2 Providence (+2) Newington
Northampton 4 2 Providence (+2) Newington
Northampton 4 2 Providence (+2)
3 Newington
Jamaica Plain Northampton
4 1 2
Providence (+2) Newington
Northampton Worcester
4 2 2
Providence (+2) Newington
Northampton Haverhill
4 2 2
Providence (+2)
60
1 Newington 4 Providence (+1) Springfield 4 Providence (+1) Springfield 4 Providence (+1)
2 Togus Northampton
3 4 Providence (+1) Togus
Springfield 3 4 Providence (+1) Togus
Springfield 3 4 Providence (+1)
3 Togus
Northampton WRJ
3 4 1
Providence (+1) Togus
Springfield Portland
2 4 2
Providence (+1) Togus
Springfield Saco
3 4 2
Providence (+1)
90
1 Northampton 4 Providence (+2) Springfield 4 Providence (+2) Springfield 4 Providence (+2)
2 Togus Northampton
3 4 Providence (+2) Togus
Springfield 3 4 Providence (+2) Togus
Springfield 3 4 Providence (+2)
3 Togus
Northampton WRJ
3 4 1
Providence (+2) Togus
Springfield Burlington
3 4 2
Providence (+2) Springfield
Bangor Littleton
4 3 4
Providence (+1)
Table 4-8 summarizes the corresponding “allocation” results; that is, the assignment of patients
to facilities (or fee basis) based on home zip code. As shown, even though the optimal locations
change for each scenario, zip codes are clustered in similar manners; that is, similar groups of
demand nodes are assigned to the same facilities. Most patients living in Vermont also become
fee type visits, which follows intuition given that demand is relatively low in Vermont as
compared to other regions. Figure 4-15 visualizes the demand nodes covered within-network (in-
house type) and outside-network (fee type) as well as the VA facility locations that provide
service assuming a 60-mile acceptable maximum travel distance and the candidate facilities
given by case 3.
Chapter 4 - Network Optimization Models
125
Table 4-8 Allocation of demand nodes to facilities for single-period model
S=60 P=3 Facilities Assigned demand node 3-digit zip codes
Case 1
West Haven 064-065-066-067-068-069 (CT) Togus 041-042-043-045-048-049 (ME)
West Roxbury 015-016-017-019-020-021-022-023-024-027 (MA), 029 (RI) Northampton 060-061 (CT), 010-011-012-013-053 (MA) Manchester 014-018 (MA), 030-031-032-033-034 (NH), 039 (ME) Providence 062-063 (CT), 028 (MA)
WRJ 037 (NH), 050-051-057 (VT) Fee type 025-026 (MA), 035-036-038 (NH), 040-044-046-047 (ME) 052-054-056-058-059 (VT)
Case 2
West Haven 064-065-066-067-068-069 (CT) Togus 042-045-048-049 (ME)
West Roxbury 015-016-017-019-020-021-022-023-024-027 (MA), 029 (RI) Manchester 014-018 (MA), 030-031-032-033-034 (NH) Providence 062-063 (CT), 028 (RI) Springfield 060-061 (CT), 010-011-012-013 (RI)
Portland 038-039-040-044 (ME) Fee type 025-026 (MA), 035-036-037 (NH), 041-044-046-047 (ME), 050-051-052-053-054-056-057-058-059 (VT)
Case 3
West Haven 064-065-066-067-068-069 (CT) Togus 042-043-045-048-049 (ME)
West Roxbury 015-016-017-019-020-021-022-023-024-027 (MA), 029 (RI) Manchester 014-018 (MA), 030-031-032-033-034 (NH) Providence 062-063 (CT), 028 (MA) Springfield 060-061 (CT), 010-011-012-013 (MA)
Saco 038-039-040-041 (ME) Fee type 025-026 (MA), 035-036-037 (NH), 044-046-047 (ME), 050-051-052-053-054-056-057-058-059 (VT)
Chapter 4 - Network Optimization Models
126
Figure 4-15 Demand nodes covered within-network and outside network and VA facilities that provide service for single-period model assuming a 60-mile acceptable maximum distance and candidate facilities under case 3
4.3.3.3 Multi-Period Results
The multi-period model was solved for five, ten, and twenty-five year planning horizons. For the
5-year case, Table 4-9 summarizes general results, again assuming a 60-mile acceptable
maximum travel distance. Since demand is expected to decrease over time, the optimal solution
Chapter 4 - Network Optimization Models
127
adds additional facilities only in the first year, although this must not necessarily be the case in
general.
Table 4-9 Results for multi-period model with 60-mile maximum acceptable travel distance
Case Num. add. facilities
(P)
Average annual cost
In-house type visit
cost
Fee type visit cost
% in-house type
visits
Optimal facility locations (total number of beds)** Savings
1 (7
candidate locations)
Est. obs.* $7,455,189 $3,252,421 $4,202,768 51% - -
0 $6,960,730 $3,305,465 $3,392,237 58% - $494,459
1 $6,734,157 $3,929,943 $2,513,087 69% Northampton(3) $721,032
2 $6,587,880 $4,360,938 $1,920,172 76% Togus(2)-Northampton(3) $867,309
3 $6,517,772 $4,562,840 $1,635,157 80% Togus(2)-Northampton(3)-WRJ(1) $937,417
2 (12
candidate locations)
Est. obs.* $7,455,189 $3,252,421 $4,202,768 51% - -
0 $6,960,730 $3,305,465 $3,392,237 58% - $494,459
1 $6,723,166 $3,914,317 $2,516,278 69% Springfield(3) $732,023
2 $6,576,890 $4,345,313 $1,923,362 76% Togus(2)-Springfield(3) $878,299
3 $6,489,340 $4,709,745 $1,416,678 82% Togus(2)-Springfield(3)-Portland(2) $965,849
3 (52
candidate locations)
Est. obs.* $7,455,189 $3,252,421 $4,202,768 51% - -
0 $6,960,730 $3,305,465 $3,392,237 58% - $494,459
1 $6,723,166 $3,914,317 $2,516,278 69% Springfield(3) $732,023
2 $6,574,090 $4,346,015 $1,920,952 76% Togus(2)-Springfield(3) $878,299
3 $6,489,340 $4,709,745 $1,416,678 82% Togus(2)-Springfield(3)-Portland(2) $965,849 *Estimated observed case **For all cases, optimum solution also requires a capacity expansion of one bed at the current facility in Providence.
As above, Figure 4-16 shows the tradeoff frontier between acceptable travel distance, average
annual cost, and coverage percentage assuming sleep beds can be added to zero through four
additional facilities and the candidate facilities given by case 3. Due to excessive computational
times, the multi-period model was solved only for maximum coverage distances of 60 miles or
less. Similar to the single-period results, the current situation also is shown for comparison. Even
with the “zero” case, an improvement of 6.6% in annual cost and 12.1% in access is possible.
Increasing the number of new facilities in which sleep beds could be added to three could result
in a savings of $965,849 (12.9%) given the same access coverage, a reduction in maximum
driving distance to 42 miles given the same cost, or any pairwise cost-distance point on the
Chapter 4 - Network Optimization Models
128
shown curve. After service capacity is added in three new facilities, again, the cost-access
tradeoff starts to become negligible.
Figure 4-16 Optimized tradeoff between acceptable distance and (a) average annual cost, and (b) coverage percentage for multi-period model assuming case 3
Table 4-10 summarizes the optimal locations and their corresponding capacities for 27 different
scenarios as previously. Similar to the single-period case, capacity expansion at the current
Providence sleep laboratory always is suggested regardless of assumptions about acceptable
$6,400
$6,750
$7,100
$7,450
$7,800
$8,150
$8,500
$8,850
$9,200
20 30 40 50 60
Cost
(in
thou
sand
s)
Maximum acceptable travel distance (S) (miles)
(a) Average annual cost
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
20 30 40 50 60
Cove
rage
Maximum acceptable travel distance (S) (miles)
(b) Coverage percentage
P=0 P=1 P=2 P=3 P=4
Current cost
Current
P=0 P=1 P=2 P=3 P=4
Chapter 4 - Network Optimization Models
129
travel distances (greater than 20 miles), with facilities Newington, Togus, Bedford, and
Springfield again being common additional expansion candidates. The broad similarity of these
results under a variety of assumptions and to those of the single-period model has been useful for
management to better understand obvious and robust decisions.
Table 4-10 Optimal facility locations for multi-period model(numbers in parentheses represent the number of additional beds in corresponding facilities)
S (miles) P
Case 1 Case 2 Case 3
Facility Beds Capacity expansion Facility Beds Capacity
expansion Facility Beds Capacity expansion
20
1 Newington 2 - Newington 2 - Newington 2 -
2 Newington Bedford
2 1 - Newington
Bedford 2 1 - Newington
Bedford 2 1 -
3 Newington
Bedford Northampton
2 1 1
- Newington
Bedford Springfield
2 1 2
- Newington
Bedford Springfield
2 1 2
-
40
1 Newington 4 Providence (+1) Newington 4 Providence (+1) Newington 4 Providence (+1)
2 Newington Bedford
4 1 Providence (+1) Newington
Springfield 2 3 Providence (+1) Springfield
Togus 3 3 Providence (+1)
3 Newington
Bedford Northampton
2 1 2
Providence (+1) Newington (+1)
(year 2)
Newington Bedford
Springfield
2 1 3
Providence (+1) Springfield Fitchburg
Togus
3 1 3
Providence (+1)
60
1 Northampton 3 Providence (+1) Springfield 3 Providence (+1) Springfield 3 Providence (+1)
2 Togus Northampton
2 3 Providence (+1) Togus
Springfield 2 3 Providence (+1) Togus
Springfield 2 3 Providence (+1)
3 Togus
Northampton WRJ
2 3 1
Providence (+1) Togus
Springfield Portland
2 3 2
Providence (+1) Togus
Springfield Portland
2 3 2
Providence (+1)
Table 4-11 summarizes patient-facility assignments (based on home zip code) for the multi-
period case, which again are very similar to those for the single-period model. One noticeable
difference is that the capacity expansions in the multi-period model are relatively lower than
those in the single-period model, which makes intuitive sense given shrinking veteran population
projections. This also results in the optimal number of patients that receive fee type service being
Chapter 4 - Network Optimization Models
130
higher especially in earlier years, contrary to current policy, with more patients who live outside
of Vermont now also being sent to non-VA facilities.
Table 4-11 Allocation of demand nodes to facilities for multi-period model
S=60 P=3 Facilities Assigned demand node 3-digit zip codes
Case 1
West Haven 061-064-065-066-067-068-069 (CT) Togus 041-042-043-045-048-049 (ME)
West Roxbury 015-016 (year 1-3)-017-018 (year 5-6)-019-020-021-022-023-024-027 (MA); 029 (RI) Northampton 010-011-012-013 (MA); 053 (VT); 060 (CT) Manchester 014-018 (year 1-4) (MA); 030-031-032-033-034 (NH); 039 (ME) Providence 016 (year 4-6) (MA); 028 (RI); 062-063 (CT)
WRJ 036-037 (NH); 050-051-057 (VT) Fee type 025-026 (MA); 035-038 (NH); 040-044-046-047 (ME); 052-054-056-058-059 (VT)
Case 2
West Haven 060 (year 3)-061-064 (year 1-2,4-6)-065-066-067-068-069 (CT) Togus 042-043 (year 4-6)-045 (year 1-3)-048-049 (ME)
West Roxbury 015-016 (year 1-3)-017-018 (year 5-6)-019-020-021-022-023-024-027 (MA); 029 (RI) Manchester 014-018 (year 1-4) (MA); 030-031-032-033-034 (NH); 039 (year 2) (ME) Providence 016 (year 4-6) (MA); 028 (RI); 062-063 (CT) Springfield 010-011-012-013 (MA); 060 (year 1-2,4-6)-064 (year 3) (CT)
Portland 038 (NH); 039 (year 1,3-6)-040-041-043 (year 1-3)-045 (year 4-6) (ME) Fee type 025-026 (MA); 035-036-037 (NH); 044-046-047 (ME); 050-051-052-053-054-056-057-058-059 (VT)
Case 3
West Haven 060 (year 3)-061-064 (year 1-2,4-6)-065-066-067-068-069 (CT) Togus 042-043 (year 4-6)-045 (year 1-3)-048-049 (ME)
West Roxbury 015-016 (year 1-3)-017-018 (year 5-6)-019-020-021-022-023-024-027 (MA); 029 (RI) Manchester 014-018 (year 1-4) (MA); 030-031-032-033-034 (NH); 039 (year 2) (ME) Providence 016 (year 4-6) (MA); 028 (RI); 062-063 (CT) Springfield 010-011-012-013 (MA); 060 (year 1-2,4-6)-064 (CT)
Portland 038 (NH); 039 (year 1,3-6)-040-041-043 (year 1-3)-045 (year 4-6) (ME) Fee type 025-026 (MA); 035-036-037 (NH); 044-046-047 (ME); 050-051-052-053-054-056-057-058-059 (VT)
In addition to the 5 year planning horizon, 10 and 25 year horizons also were considered. To
reduce execution times for the 10 and 25 year horizon cases, increments of two-year and five-
year periods were employed respectively, with the projected demand for each n-year period
aggregated accordingly (that is, each model still used 5 periods, for example with the 25 year
Chapter 4 - Network Optimization Models
131
model treating the first 5 years as one period, the second 5 years as a second period, and so on).
The average annual cost and coverage percentage for each planning horizon assuming sleep beds
can be added to at most three additional facilities and the candidate facilities given by case 3 are
shown in Figure 4-17. Of note, as the planning horizon increases the average annual cost
decreases, primarily because of long-term decreases in demand.
Figure 4-17 Optimal (a) annual cost and (b) coverage percentages considering different planning horizons (both for case 3 and P = 3)
$6,000
$6,500
$7,000
$7,500
$8,000
$8,500
$9,000
20 30 40 50 60
Cost
per
yea
r (in
thou
sand
s)
Maximum acceptable travel distance (S) (miles)
(a) Average annual cost
20%
30%
40%
50%
60%
70%
80%
90%
20 30 40 50 60
Cove
rage
Maximum acceptable travel distance (S) (miles)
(b) Coverage percentage
5 year planning horizon 10 year planning horizon 25 year planning horizon
Chapter 4 - Network Optimization Models
132
4.3.4 Sensitivity Analysis on Demand Variability
The above mathematical models assume that demand for VA specialty care services is
deterministic and equally distributed over time. Although assumption of deterministic demand
significantly decreases the complexity of a mathematical model, it may not always be justified.
Given the future veteran population uncertainty due to unpredictable military conflicts,
additional research was conducted to study the impact of stochastic demand on the deterministic
model results. Two simulation models, as explained in Figure 4-18, were developed that consider
demand variability on a yearly and monthly basis. Assuming the optimal locations and capacities
that resulted from the “single period model for case 3 and P = 3”, the changes in the overall
system results were examined assuming variable annual and monthly demands.
Figure 4-18 Pseudo code for the simulation models with demand variability
Prompt user for desired coefficient of variation (CV) and half-width values (desired HW) Initializations (outside-replications) Do While HW > desired HW
Initializations (within-replications) FOR i = 1 to 59 !for each demand node
Generate demandi ~ NORMAL(μi, σi) NEXT FOR j = 1 to 7 !for each facility
FOR i = 1 to 59 IF demand node i is assigned to facility j
IF capacity j is not exceeded Let house_cost = house_cost + unit_house_cost*demandi
ENDIF IF capacity j is exceeded
Let fee_cost = fee_cost + unit_fee_cost*demandi Let unanticipated_fee = unanticipated_fee + demandi Let noncovered = noncovered + demandi
ENDIF ENDIF IF demand node i is assigned to a fee facility
Let fee_cost = fee_cost + unit_fee_cost*demandi Let noncovered = noncovered + demandi
ENDIF NEXT
NEXT Calculate HW
ENDDO Calculate average and standard deviation of costs, unanticipated fee patients, coverage percentage Print results END
If monthly demand considered, add: For k = 1 to 12 !for each month
Generate demandik ~ NORMAL(μi/12, √σi
2/12)
Chapter 4 - Network Optimization Models
133
Figure 4-19 illustrates the changes in average total, house, and fee costs and number of
unanticipated fee patients (i.e., patients that are sent to a VA facility by the deterministic model
but cannot be served there due to capacity constraints) assuming variable annual demands. As
expected, the number of unanticipated fee patients significantly increases as the coefficient of
variation (CV) gets larger, resulting in higher fee cost and thus total cost. The average house
cost, on the other hand, decreases as the CV increases, mainly because of the capacity constraint.
Due to the limited capacity, as demand varies, house cost cannot increase more than an upper
limit (i.e., where all facilities are fully utilized), while there is no lower limit. Since all facilities
are almost fully utilized in the no-variation experiments (i.e., CV = 0), as the CV increases,
positive deviations has no/limited effects on the house cost while negative deviations pull it
down drastically. This causes the average house cost to be lower than the one with the
deterministic demand.
As shown in Figure 4-20, the number of unanticipated fee patients assuming variable monthly
demands is significantly higher than those with the annual demand variability. This is because
the monthly variation is higher than the annual variation assuming the same annual CV values,
which results in less ability to meet all demands within-network. Accordingly, increases in the
average total cost, number of unanticipated fee patients, and related fee cost and decreases in the
average house cost are more significant in the monthly demand variability case when compared
to those with the variable annual demand.
Overall, this analysis suggests that accounting for demand variability makes significant changes
in the performance and may change the system design completely. Thus, stochastic modeling
approaches may be beneficial even though they increase mathematical complexity significantly.
Chapter 4 - Network Optimization Models
134
Figure 4-19 Changes in average (a) total cost, (b) number of unanticipated fee patients, (c) house cost, and (d) fee cost assuming variable annual demands and single period model results for case 3 and P = 3
$0
$2,000
$4,000
$6,000
$8,000
$10,000
$12,000
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Cost
(in
thou
sand
s)
Coefficient of variation
(a) Average total cost
Minimum
Maximum
95% prob. range
Total cost with deterministic demand
0
700
1,400
2,100
2,800
3,500
4,200
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Num
ber o
f pat
ient
s
Coefficient of variation
(b) Average number of unanticipated fee patients
Maximum
Minimum
95% prob. range
$0
$1,000
$2,000
$3,000
$4,000
$5,000
$6,000
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Cost
(in
thou
sand
s)
Coefficient of variation
(c) Average house cost
Maximum
Minimum
95% prob. range
House cost with deterministic demand
$0
$1,000
$2,000
$3,000
$4,000
$5,000
$6,000
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Cost
(in
thou
sand
s)
Coefficient of variation
(d) Average fee cost
95% prob. range Maximum
Minimum Fee cost with
deterministic demand
Chapter 4 - Network Optimization Models
135
Figure 4-20 Changes in average (a) total cost, (b) number of unanticipated fee patients, (c) house cost, and (d) fee cost assuming variable monthly demands and single period model results for case 3 and P = 3
$0
$2,000
$4,000
$6,000
$8,000
$10,000
$12,000
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Cost
(in
thou
sand
s)
Coefficient of variation
(a) Average total cost
Minimum
Maximum
95% prob. range
Total cost with deterministic demand
0
700
1,400
2,100
2,800
3,500
4,200
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Num
ber o
f pat
ient
s
Coefficient of variation
(b) Average number of unanticipated fee patients
Maximum
Minimum
95% prob. range
$0
$1,000
$2,000
$3,000
$4,000
$5,000
$6,000
0.00 0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40
Cost
(in
thou
sand
s)
Coefficient of variation
(c) Average house cost
Maximum
Minimum
95% prob. range
House cost with deterministic demand
$0
$1,000
$2,000
$3,000
$4,000
$5,000
$6,000
0 0.04 0.08 0.12 0.16 0.2 0.24 0.28 0.32 0.36 0.4
Cost
(in
thou
sand
s)
Coefficient of variation
(d) Average fee cost
95% prob. range
Maximum
Minimum Fee cost with
deterministic demand
Chapter 4 - Network Optimization Models
136
4.3.5 Discussion
As the U.S. veteran population ages, effective delivery of health services becomes more
important in terms of optimal within-network coverage at minimum total cost and travel
distance. This study investigated the optimal short and long term location of sleep apnea testing
services across New England VA facilities. Results indicate that significant cost savings and
improvements in patient-centeredness can be achieved simply by optimizing capacity in existing
VA facilities. These potential savings demonstrate this is a useful approach to help decision
makers make more deliberate and tactical healthcare service location and capacity optimization
decisions. In this particular application, significant savings are possible by opening an additional
sleep laboratory to serve unmet need in northern Connecticut and western Massachusetts. If
closing or relocating underutilized facilities were allowed, even greater improvements could be
achieved, especially in the multi-period case. These results also have implications on the optimal
amount of out-of-network “fee” care that run contrary to current practice and beliefs. Counter to
the VA’s desire to eliminate almost all fee basis care, in almost all specialty care applications
and scenarios examined to-date, roughly 10 to 25% of all care is optimal to occur outside of the
VA network. To better understand the costs of out-of-network care, the above models were
modified to require that all patients must be covered by VA facilities, resulting in a 34% increase
in average travel distance while decreasing the total cost only by 4% (versus roughly 5% when
compared with the S = 60 and P = 3 case).
Possible limitations of this study include the assumption that out of network capacity always is
available within desired distances, which may not reflect the actual market. Additional savings
might be achieved by adding or relocating existing facilities. Expansion of the system is perhaps
Chapter 4 - Network Optimization Models
137
less disruptive than facility relocation, the latter potentially forcing staff to relocate or find work
elsewhere. Since the disruptive nature of change may limit a health system’s ability to
immediately implement an optimized network design, some thought might be put into multi-
period transition or minimally disruptive models.
These models could be expanded in several manners. These include accounting for demand
variability naturally occurring over time and future veteran population uncertainty due to such
things as the unpredictability of future military conflicts. Stochastic modeling approaches might
be appropriate here, such as recourse, chance-constraint, and dependent-chance programs. Multi-
period capacity scale-down models also might be considered, although implementation initially
may be limited by “permanent” contracts and the ability to retrain staff. Finally, implementation
of these models in user-friendly decision support tools also would be useful to enable decision
makers to explore what-if scenarios more readily.
Chapter 5 - Conclusions and Future Work
138
Conclusions and Future Work Chapter 5 -
This study focuses on developing systems engineering models to improve the effectiveness and
cost of and access to screening, diagnostic, and care services for several mental/general health
conditions within large integrated healthcare systems. The use of several systems engineering/
operations research modeling techniques were illustrated within the Veterans Health
Administration (VHA) to discover potential improvement opportunities in providing effective
and timely care to the veterans with significant health problems while considering overall system
cost. Six mathematical models – probability modeling, Monte Carlo, fuzzy logic, logistic
regression, artificial neural networks, and integer programming – were developed and illustrated
across a range of specialty care services, namely, traumatic brain injury (TBI), posttraumatic
stress disorder (PTSD), and sleep apnea screening, diagnostic, and treatment services. Results
indicate that these models could make a significant contribution to the safety and long-term well-
being of U.S. servicemen. For each specific model, detailed results, limitations, and possible
extensions are provided at the end of each chapter.
As a summary, the first chapter of this study investigated the current and proposed screening
processes for PTSD within the VHA and further explored the effects of several model parameters
on the number of false-diagnoses and overall cost. Results indicate that lower false diagnosis
Chapter 5 - Conclusions and Future Work
139
rates, predictable performance, and reduced costs can be achieved by using a more intentionally
designed system. Also developed was a sequential screening model for mild TBI to illustrate
how the general approach can be applied to other disorders. The biggest limitation of this study
is that the results are very dependent on the model inputs, most of which are not known with
certainty. One possible extension, therefore, is to conduct further research to better estimate these
inputs. Additionally, the proposed screening process for mild TBI could be further developed
and evaluated by better identifying the model parameters and by conducting a comprehensive
sensitivity analysis on the inputs that are subject to uncertainty.
In the second chapter of this study, a comprehensive diagnostic model which aims to ensure
proper and timely diagnosis of TBI was developed. Three types of mathematical methods were
used to categorize TBI patients into the most probable severity states, with preliminary results
indicating a fairly strong agreement with the patients’ known health states for each method. This
study could be extended in a number of ways, one of them being to develop mathematical
models to optimize overall or severity-specific performance of the proposed model.
Finally, the last chapter illustrated the use of location-allocation modeling approach across a
range of specialty care services, namely, PTSD treatment and sleep apnea testing services, within
the VHA. In both applications, results indicate that significant cost savings and improvements in
patient-centeredness can be achieved. These potential savings demonstrate that this is a useful
approach to help decision makers make more informed and tactical decisions. General
limitations include the assumptions regarding model inputs and deterministic demand. The
mathematical models could be extended in a manner so as to account for demand variability and
uncertainty. Another extension could be to develop a co-location model which simultaneously
Chapter 5 - Conclusions and Future Work
140
determines the optimal locations and capacities for multiple mental health services (e.g., PTSD
and depression care services), given that the mental health conditions often co-occur and interact
with each other.
More generally, the results of this study underscore three important issues: (1) current care
systems are insufficient to meet the mental health and cognitive needs of U.S. service members
and produce extra costs, (2) significant opportunities for mental health care process
improvements and cost reductions are possible with the use of systems engineering methods such
as statistical modeling and mathematical optimization, and (3) further improvements in the long-
term care of U.S. servicemen can be achieved by applying these models over a range of other
chronic health conditions. However, there exist several implementation challenges given that the
results do not consider the potential for reduced care continuity and that the disruptive nature of
change may limit a health system’s ability to immediately implement an optimized system
design.
The mathematical models described in this study all share another related potential limitation
which is poor “trialability”, especially given that healthcare practitioners have become more
accustomed to gradual iterative PDSA improvement cycles than to instantaneous reengineering.
Small cycles of change allow process improvement personnel to obtain safe feedback to minor
changes before proceeding with broader implementation. Solutions such as those in this study,
conversely, may require more of a leap of faith. Unfortunately, since few healthcare practitioners
and leaders understand modeling, simulation, and optimization methods, it may be difficult for
them to fully trust in results and make disruptive changes.
Chapter 5 - Conclusions and Future Work
141
Although the potential improvements that can be achieved with the use of each mathematical
model developed in this study are illustrated separately, it is important to emphasize that these
models and their results highly interact with each other. For instance, the screening and
diagnostic models determine the number of patients in need of using care services in a
population, which actually constitutes the true demand for care services – one of the inputs in the
network optimization models. Integration of these models in a manner so as to use one model’s
output as another’s input, therefore, could help identify the correct inputs for each model and
thus provide more accurate results.
In conclusion, there are a number of possibilities to extend this study, all of which are discussed
throughout the dissertation. These recommendations are summarized as follows:
(1) Given that the results are very dependent on the model inputs, additional research could
be conducted either to confirm the estimates used in this study or to make better
estimates. Integration of mathematical models and using one’s output as another’s input
also could help identify the inputs more accurately.
(2) Mathematical models could be extended in order to account for (i) demand variability
naturally occurring over time and (ii) future veteran population uncertainty due to the
unpredictability of future military conflicts, using stochastic modeling approaches such as
recourse, chance-constraint, and dependent-chance programs.
(3) In order to address the difficulties in implementation and the poor trialability, multi-
period transition or minimally disruptive models could be considered. The development
Chapter 5 - Conclusions and Future Work
142
of graphical and interactive tools also could be useful for building initial acceptance,
understanding, support, and trust.
(4) Finally, implementation of these models in user-friendly decision support tools would be
useful to enable decision makers to explore what-if scenarios more readily.
Bibliography
143
Bibliography
Aboolian, R., Sun, Y., and Koehler, G.J., 2009, "A location-allocation problem for a web services provider in a competitive market", European Journal of Operational Research, 194(1): 64-77.
Adenso-Díaz, B. and Rodríguez, F., 1997, "A simple search heuristic for the MCLP: Application to the location of ambulance bases in a rural region", Omega, 25(2): 181-187.
American Academy of Sleep Medicine, 1999, "Sleep-related breathing disorders in adults: Recommendations for syndrome definition and measurement techniques in clinical research", Sleep, 22(5): 667-689.
American Psychiatric Association, 2000, Diagnostic and statistical manual of mental disorders : DSM-IV-TR, 4th ed, Washington, DC.
American Sleep Apnea Association, 2012a, Obstructive sleep apnea treatment options, http://www.sleepapnea.org/treat/treatment-options.html [accessed online 08/09/2013].
American Sleep Apnea Association, 2012b, Sleep apnea, http://www.sleepapnea.org/learn/sleep-apnea.html [accessed online 08/09/2013].
Amin, M.M., Parisi, J.A., Gold, M.S., and Gold, A.R., 2010, "War-related illness symptoms among Operation Iraqi Freedom/Operation Enduring Freedom returnees", Military Medicine, 175(3): 155-157.
Anagnostou, T., Remzi, M., Lykourinas, M., and Djavan, D., 2003, "Artificial neural networks for decision-making in urologic oncology", European Urology, 43(6): 596-603.
Ancoli-Isreal, S., Kripke, D.F., Klauber, M.R., Mason, W.J., Fell, R., and Kaplan, O., 1991, "Sleep-disordered breathing in community-dwelling elderly", Sleep, 14(6): 486-495.
Ancoli-Isreal, S. and Avalon, L., 2006, "Diagnosis and treatment of sleep disorders in older adults", American Journal of Geriatric Psychiatry, 14(2): 95-103.
Anderson-Barnes, V.C., Weeks, S.R., and Tsao, J.W., 2010, "Mild traumatic brain injury update", Continuum: Lifelong Learning in Neurology, 16(6): 17-26.
Bibliography
144
Anderson, G.F., Reinhardt, U.E., Hussey, P.S., and Petrosyan, V., 2003, "It’s the prices, stupid: Why the United States is so different from other countries", Health Affairs, 22(3): 89-105.
Asemota, A.O., George, B.P., Bowman, S.M., Haider, A.H., and Schneider, E.B., 2013, "Causes and trends in traumatic brain injury for United States adolescents", Journal of Neurotrauma, 30(2): 67-75.
Asgary, M.P., Jahandideh, S., Abdolmaleki, P., and Kazemnejad, A., 2007, "Analysis and identification of β-turn types using multinomial logistic regression and artificial neural network", Bioinformatics, 23(23): 3125-3130.
Ayer, T., Alagoz, O., and Stout, N.K., 2012, "OR forum: A POMDP approach to personalize mammography screening decisions", Operations Research, 60(5): 1019-1034.
Baker, R.D., 1998, "Use of a mathematical model to evaluate breast cancer screening policy", Health Care Management Science, 1(2): 103-113.
Barnwell, S.V., Juretic, M.A., Hoerster, K.D., Van de Plasch, R., and Felker, B.L., 2012, "VA Puget Sound Telemental Health Service to rural veterans: A growing program", Psychological Services, 9(2): 209-211.
Beninati, W. and Sanders, M.H., 2001, "Optimal continuous positive airway pressure for the treatment of obstructive sleep apnea/hypopnea", Sleep Medicine Reviews, 5(1): 7-23.
Benneyan, J. and Kaminsky, F., 1996, "Statistical and economic models for analysis and optimal design of laboratory screening policies for cervical cancer", Annals of Operations Research, 67(1): 235-285.
Bisson, J.I., 2007, "Post-traumatic stress disorder", British Medical Journal, 334(7597): 789-793.
Blake, D.D., Weathers, F., Nagy, L.M., Kaloupek, D.G., Klauminzer, G., Charney, D.S., and Keane, T.M., 1990, "A clinician rating scale for assessing current and lifetime PTSD: The CAPS-1", Behavior Therapist, 13: 187-188.
Blake, D.D., Weathers, F.W., Nagy, L.M., Kaloupek, D.G., Gusman, F.D., Charney, D.S., and Keane, T.M., 1995, "The development of a clinician-administered PTSD scale", Journal of Traumatic Stress, 8(1): 75-90.
Boscarino, J.A., Larson, S., Ladd, I., Hill, E., and Paolucci, S.J., 2010, "Mental health experiences and needs among primary care providers treating OEF/OIF veterans: Preliminary findings from the Geisinger Veterans Initiative", International Journal of Emergency Mental Health, 12(3): 161-170.
Branas, C.C., MacKenzie, E.J., and ReVelle, C.S., 2000, "A trauma resource allocation model for ambulances and hospitals", Health Services Research, 35(2): 489-507.
Bibliography
145
Brewin, C.R., Andrews, B., and Valentine, J.D., 2000, "Meta-analysis of risk factors for posttraumatic stress disorder in trauma-exposed adults", Journal of Consulting and Clinical Psychology, 68(5): 748-766.
Brundtland, G.H., 2000, "Mental health in the 21st century", Bulletin of the World Health Organization, 78(4): 411.
Bruni, M., Conforti, D., Sicilia, N., and Trotta, S., 2006, "A new organ transplantation location–allocation policy: A case study of Italy", Health Care Management Science, 9(2): 125-142.
Butler, D., Buono, J., Erdtmann, F., and Reid, P., 2008, Systems engineering to improve traumatic brain injury care in the military health system, Washington, DC: The National Academies Press.
Champion, H.R., Holcomb, J.B., and Young, L.A., 2009, "Injuries from explosions: physics, biophysics, pathology, and required research focus", Journal of Trauma, 66(5): 1468-1477.
Chan, Y.H., 2005, "Biostatistics 305. multinomial logistic regression", Singapore Medical Journal, 46(6): 259-268.
Chang, C.L. and Chen, C.H., 2009, "Applying decision tree and neural network to increase quality of dermatologic diagnosis", Expert Systems with Applications, 36(2): 4035-4041.
Church, R.L. and ReVelle, C., 1974, "The maximal covering location problem", Papers in Regional Science, 32(1): 101-118.
Church, R.L., 1984, "The planar maximal covering location problem", Journal of Regional Science, 24(2): 185-201.
Ciechanowski, P.S., Katon, W.J., and Russo, J.E., 2000, "Depression and diabetes: Impact of depressive symptoms on adherence, function, and costs", Archives of Internal Medicine, 160(21): 3278-3285.
Cinar, M., Engin, M., Engin, E.Z., and Atesci, Y.Z., 2009, "Early prostate cancer diagnosis by using artificial neural networks and support vector machines", Expert Systems with Applications, 36(3): 6357-6361.
Cloitre, M., Courtois, C.A., Ford, J.D., Green, B.L., Alexander, P., Briere, J., Herman, J.L., Lanius, R., Stolbach, B.C., Spinazzola, J., Van der Kolk, B.A., Van der Hart, O., 2012, The ISTSS Expert Consensus Treatment Guidelines for Complex PTSD in Adults, http://www.istss.org/AM/Template.cfm?Section=ISTSS_Complex_PTSD_Treatment_Guidelines&Template=/CM/ContentDisplay.cfm&ContentID=5185 [accessed online 07/28/2013].
Cohen, B.E., Gima, K., Bertenthal, D., Kim, S., Marmar, C.R., and Seal, K.H., 2010, "Mental health diagnoses and utilization of VA non-mental health medical services among returning Iraq and Afghanistan veterans", Journal of General Internal Medicine, 25(1): 18-24.
Bibliography
146
Colton, C.W. and Manderscheid, R.W., 2006, "Congruencies in increased mortality rates, years of potential life lost, and causes of death among public mental health clients in eight states", Preventing Chronic Disease, 3(2): A42.
Congress of the United States, Congressional Budget Office, 2012, The Veterans Health Administration’s Treatment of PTSD and Traumatic Brain Injury Among Recent Combat Veterans, http://www.cbo.gov/sites/default/files/cbofiles/attachments/02-09-PTSD.pdf [accessed online 08/03/2013].
Cooper, L., 1963, "Location-allocation problems", Operations Research, 11(3): 331-343.
Côté, M.J., Syam, S.S., Vogel, W.B., and Cowper, D.C., 2007, "A mixed integer programming model to locate traumatic brain injury treatment units in the Department of Veterans Affairs: A case study", Health Care Management Science, 10(3): 253-267.
Crisp, A.H., Gelder, M.G., Rix, S., Meltzer, H.I., and Rowlands, O.J., 2000, "Stigmatisation of people with mental illnesses", British Journal of Psychiatry, 177: 4-7.
Current, J. and O'Kelly, M., 1992, "Locating emergency warning sirens", Decision Sciences, 23(1): 221-234.
Czabanski, R., Jezewski, J., Wrobel, J., Sikora, J., and Jezewski, M., 2013, "Application of fuzzy inference systems for classification of fetal heart rate tracings in relation to neonatal outcome", Ginekologia Polska, 84(1): 38-43.
Dalalah, D. and Magableh, S., 2008, "A remote fuzzy multicriteria diagnosis of sore throat", Telemedicine Journal and e-Health, 14(7): 656-665.
Daskin, M. and Dean, L., 2005, Location of health care facilities, in Handbook of OR/MS in Health Care: A Handbook of Methods and Applications, F. Sainfort, M. Brandeau, and W. Pierskalla, Editors, Boston, MA: Kluwer, p. 43-76.
Davis, L.L., Leon, A.C., Toscano, R., Drebing, C.E., Ward, L.C., Parker, P.E., Kashner, T.M., and Drake, R.E., 2012, "A randomized controlled trial of supported employment among veterans with posttraumatic stress disorder", Psychiatric Services, 63(5): 464-470.
Defense and Veterans Brain Injury Center (DVBIC), 2013, Medical diagnoses of TBI in the United States since 2000, DoD Worldwide Numbers for TBI, http://www.dvbic.org/sites/default/files/u9/dod-tbi-worldwide-2000-2013Q1-as-of-130509.pdf [accessed online 06/30/2013].
Defense and Veterans Brain Injury Center (DVBIC), 2013, TBI basics, http://www.dvbic.org/tbi-basics [accessed online 06/30/2013].
Dhawan, P., Rose, A., Krassioukov, A., and Miller, W.C., 2006, "Early interventions for mild traumatic brain injury: Reflections on experience", BC Medical Journal, 48(9): 442-446.
Bibliography
147
Donner, A., Young, C., and Bass, M., 1979, "Sequential screening for hypertension in primary care", Journal of Chronic Diseases, 32(8): 577-581.
Eaton, D.J., Daskin, M.S., Simmons, D., Bulloch, B., and Jansma, G., 1985, "Determining emergency medical service vehicle deployment in Austin, Texas", Interfaces, 15(1): 96-108.
El-Ad, B. and Lavie, P., 2005, "Effect of sleep apnea on cognition and mood", International Review of Psychiatry, 17(4): 277-282.
Elder, G.A., Mitsis, E.M., Ahlers, S.T., and Cristian, A., 2010, "Blast-induced Mild Traumatic Brain Injury", Psychiatric Clinics of North America, 33(4): 757-781.
Elgmark Andersson, E., Lund, J., and Mansson, J., 2010, "Traumatic brain injury in children between 7-12 years of age", Developmental Neurorehabilitation, 13(5): 346-350.
Ellen, R.L., Marshall, S.C., Palayew, M., Molnar, F.J., Wilson, K.G., and Man-Son-Hing, M., 2006, "Systematic review of motor vehicle crash risk in persons with sleep apnea", Journal of Clinical Sleep Medicine, 2(2): 193-200.
Erman, M.K., 2006, "Selected sleep disorders: Restless legs syndrome and periodic limb movement disorder, sleep apnea syndrome, and narcolepsy", Psychiatric Clinics of North America, 29(4): 947-967.
Faul, M., Xu, L., Wald, M.M., and Coronado, V.G., 2010, Traumatic brain injury in the United States: Emergency department visits, hospitalizations and deaths 2002–2006, Atlanta, GA: Centers for Disease Control and Prevention, National Center for Injury Prevention and Control.
Ferguson, K.A., Cartwright, R., Rogers, R., and Schmidt-Nowara, W., 2006, "Oral appliances for snoring and obstructive sleep apnea: A review", Sleep, 29(2): 244-262.
Foa, E.B., Keane, T.M., Friedman, M.J., and Cohen, J.A., 2008, Effective Treatments for PTSD, 2nd ed, New York, NY: Guilford Press.
Foa, E.B. and Yadin, E., 2011, "Assessment and diagnosis of posttraumatic stress disorder", Psychiatric Times, 28(7).
Forbes, D., Creamer, M., and Biddle, D., 2001, "The validity of the PTSD Checklist as a measure of symptomatic change in combat-related PTSD", Behaviour Research and Therapy, 39: 977-986.
Freedman, S.A., Gluck, N., Tuval-Mashiach, R., Brandes, D., Peri, T., and Shalev, A.Y., 2002, "Gender differences in responses to traumatic events: A prospective study", Journal of Traumatic Stress, 15(5): 407-413.
Freire, A.O., Sugai, G.C., Chrispin, F.S., Togeiro, S.M., Yamamura, Y., Mello, L.E., and Tufik, S., 2007, "Treatment of moderate obstructive sleep apnea syndrome with acupuncture: A randomised, placebo-controlled pilot trial", Sleep Medicine, 8(1): 43-50.
Bibliography
148
Gay, P., Weaver, T., Loube, D., and Iber, C., 2006, "Evaluation of positive airway pressure treatment for sleep related breathing disorders in adults", Sleep, 29(3): 381-401.
Gehi, A., Haas, D., Pipkin, S., and Whooley, M.A., 2005, "Depression and medication adherence in outpatients with coronary heart disease: Findings from the Heart and Soul Study", Archives of Internal Medicine, 165(21): 2508-2513.
George, C.F., 2007, "Sleep apnea, alertness, and motor vehicle crashes", American Journal of Respiratory and Critical Care Medicine, 176(10): 954-956.
Goetzel, R.Z., Ozminkowski, R.J., Sederer, L.I., and Mark, T.L., 2002, "The business case for quality mental health services: Why employers should care about the mental health and well-being of their employees", Journal of Occupational and Environmental Medicine, 44(4): 320-330.
Gray, M.J., Litz, B.T., Hsu, J.L., and Lombardo, T.W., 2004, "Psychometric properties of the life events checklist", Assessment, 11(4): 330-341.
Guler, I., Hardalac, F., and Barisci, N., 2002, "Application of FFT analyzed cardiac doppler signals to fuzzy algorithm", Computers in Biology and Medicine, 32(6): 435-444.
Guler, I., Tunca, A., and Gulbandilar, E., 2008, "Detection of traumatic brain injuries using fuzzy logic algorithm", Expert Systems with Applications, 34(2): 1312-1317.
Guler, I., Gokcil, Z., and Gulbandilar, E., 2009, "Evaluating of traumatic brain injuries using artificial neural networks", Expert Systems with Applications, 36(7): 10424-10427.
Haas-Wilson, D. and Savoca, E., 1990, "Quality and provider choice: A multinomial logit-least-squares model with selectivity", Health Services Research, 24(6): 791-809.
Hailey, D., Roine, R., and Ohinmaa, A., 2008, "The effectiveness of telemental health applications: A review", Canadian Journal of Psychiatry, 53(11): 769-778.
Halligan, S.L. and Yehuda, R., 2000, "Risk Factors for PTSD", PTSD Research Quarterly, 11(3): 1-7.
Haneef, Z., Levin, H.S., Frost, J.D., Jr., and Mizrahi, E.M., 2013, "Electroencephalography and quantitative electroencephalography in mild traumatic brain injury", Journal of Neurotrauma, 30(8): 653-656.
Harper, P.R., Shahani, A.K., Gallagher, J.E., and Bowie, C., 2005, "Planning health services with explicit geographical considerations: A stochastic location-allocation approach", Omega, 33(2): 141-152.
Harris, R.P., Helfand, M., Woolf, S.H., Lohr, K.N., Mulrow, C.D., Teutsch, S.M., and Atkins, D., 2001, "Current methods of the US Preventive Services Task Force: A review of the process", American Journal of Preventive Medicine, 20(3 Suppl): 21-35.
Bibliography
149
Heegaard, W. and Biros, M., 2007, "Traumatic brain injury", Emergency Medicine Clinics of North America, 25(3): 655-678.
Hermann, B.A., Shiner, B., and Friedman, M.J., 2012, "Epidemiology and prevention of combat-related post-traumatic stress in OEF/OIF/OND service members", Military Medicine, 177(8 Suppl): 1-6.
Hermes, E.D., Rosenheck, R.A., Desai, R., and Fontana, A.F., 2012, "Recent trends in the treatment of posttraumatic stress disorder and other mental disorders in the VHA", Psychiatric Services, 63(5): 471-476.
Hoge, C.W., Castro, C.A., Messer, S.C., McGurk, D., Cotting, D.I., and Koffman, R.L., 2004, "Combat duty in Iraq and Afghanistan, mental health problems, and barriers to care", New England Journal of Medicine, 351(1): 13-22.
Hoge, C.W., Auchterlonie, J.L., and Milliken, C.S., 2006, "Mental health problems, use of mental health services, and attrition from military service after returning from deployment to Iraq or Afghanistan", Journal of the American Medical Association (JAMA), 295(9): 1023-1032.
Hoge, C.W., McGurk, D., Thomas, J.L., Cox, A.L., Engel, C.C., and Castro, C.A., 2008, "Mild traumatic brain injury in U.S. soldiers returning from Iraq", New England Journal of Medicine, 358(5): 453-463.
Hossain, M., Wright, S., and Petersen, L.A., 2002, "Comparing performance of multinomial logistic regression and discriminant analysis for monitoring access to care for acute myocardial infarction", Journal of Clinical Epidemiology, 55(4): 400-406.
Institute of Medicine, 1999, To err is human: Building a safer health system, Washington, DC: The National Academies Press.
Institute of Medicine, 2001, Crossing the quality chasm: A new health system for the 21st century, Washington, DC: The National Academies Press.
Ivanova, J.I., Birnbaum, H.G., Chen, L., Duhig, A.M., Dayoub, E.J., Kantor, E.S., Schiller, M.B., and Phillips, G.A., 2011, "Cost of post-traumatic stress disorder vs major depressive disorder among patients covered by medicaid or private insurance", American Journal of Managed Care, 17(8): e314-323.
Jackson, G.L., Hamilton, N.S., and Tupler, L.A., 2008, "Detecting traumatic brain injury among veterans of Operations Enduring and Iraqi Freedom", North Carolina Medical Journal, 69(1): 43-47.
Jacobs, D.A., Silan, M.N., and Clemson, B.A., 1996, "An analysis of alternative locations and service areas of American Red Cross blood facilities", Interfaces, 26(3): 40-50.
Bibliography
150
Javouhey, E., Guérin, A., and Chiron, M., 2006, "Incidence and risk factors of severe traumatic brain injury resulting from road accidents: A population-based study", Accident Analysis & Prevention, 38(2): 225-233.
Kaminsky, F.C., Benneyan, J.C., and Mullins, D.L., 1997, "Automated rescreening in cervical cytology: Mathematical models for evaluating overall process sensitivity, specificity and cost", Acta Cytologica, 41(1): 209-223.
Katon, W., Cantrell, C.R., Sokol, M.C., Chiao, E., and Gdovin, J.M., 2005, "Impact of antidepressant drug adherence on comorbid medication use and resource utilization", Archives of Internal Medicine, 165(21): 2497-2503.
Kaufman, K.R., Brey, R.H., Chou, L., Rabatin, A., Brown, A.W., and Basford, J.R., 2006, "Comparison of subjective and objective measurements of balance disorders following traumatic brain injury", Medical Engineering & Physics, 28(3): 234-239.
Keane, T.M. and Kaloupek, D.G., 1997, "Comorbid psychiatric disorders in PTSD: Implications for research", Annals of the New York Academy of Sciences, 821(1): 24-34.
Keen, S.M., Kutter, C.J., Niles, B.L., and Krinsley, K.E., 2008, "Psychometric properties of PTSD Checklist in sample of male veterans", Journal of Rehabilitation Research & Development, 45(3): 465-474.
Kent, D.L., Shachter, R., Sox, H.C., Hui, N.S., Shortliffe, L.D., Moynihan, S., and Torti, F.M., 1989, "Efficient scheduling of cystoscopies in monitoring for recurrent bladder cancer", Medical Decision Making, 9(1): 26-37.
Kessler, R.C., McGonagle, K.A., Zhao, S., Nelson, C.B., Hughes, M., Eshleman, S., Wittchen, H.U., and Kendler, K.S., 1994, "Lifetime and 12-month prevalence of DSM-III-R psychiatric disorders in the United States: Results from the National Comorbidity Survey", Archives of General Psychiatry, 51(1): 8-19.
Kessler, R.C., Sonnega, A., Bromet, E., Hughes, M., and Nelson, C.B., 1995, "Posttraumatic stress disorder in the National Comorbidity Survey", Archives of General Psychiatry, 52(12): 1048-1060.
Kessler, R.C., Zhao, S., Katz, S.J., Kouzis, A.C., Frank, R.G., Edlund, M., and Leaf, P., 1999, "Past-year use of outpatient services for psychiatric problems in the National Comorbidity Survey", American Journal of Psychiatry, 156(1): 115-123.
Kessler, R.C., Berglund, P.A., Bruce, M.L., Koch, J.R., Laska, E.M., Leaf, P.J., Manderscheid, R.W., Rosenheck, R.A., Walters, E.E., and Wang, P.S., 2001, "The prevalence and correlates of untreated serious mental illness", Health Services Research, 36(6 Part 1): 987-1007.
Kessler, R.C., Berglund, P., Demler, O., Jin, R., Merikangas, K.R., and Walters, E.E., 2005, "Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication", Archives of General Psychiatry, 62(6): 593-602.
Bibliography
151
Kessler, R.C., Chiu, W.T., Demler, O., and Walters, E.E., 2005, "Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication", Archives of General Psychiatry, 62(6): 617-627.
Khan, M.U., Choi, J.P., Shin, H., and Kim, M., 2008, "Predicting breast cancer survivability using fuzzy decision trees for personalized health care", in IEEE Engineering in Medicine and Biology Society Conference Proceedings.
Kizer, K.W., 2012, "Veterans and the affordable care act", Journal of the American Medical Association (JAMA), 307(8): 789-790.
Kjelsberg, F.N., Ruud, E.A., and Stavem, K., 2005, "Predictors of symptoms of anxiety and depression in obstructive sleep apnea", Sleep Medicine, 6(4): 341-346.
Klose, A. and Drexl, A., 2005, "Facility location models for distribution system design", European Journal of Operational Research, 162(1): 4-29.
Knudsen, A.B., Hur, C., Gazelle, G.S., Schrag, D., McFarland, E.G., and Kuntz, K.M., 2012, "Rescreening of persons with a negative colonoscopy result: Results from a microsimulation model", Annals of Internal Medicine, 157(9): 611-620.
Kohn, R., Saxena, S., Levav, I., and Saraceno, B., 2004, "The treatment gap in mental health care", Bulletin of the World Health Organization, 82(11): 858-866.
Krose, B. and Smagt, P., 1996, An introduction to neural networks, 8th ed, Amsterdam, The Netherlands: The University of Amsterdam.
Kussman, M.J., 2008, Uniform mental health services in VA medical centers and clinics, Veterans Health Administration Handbook 1160.01, http://www.mirecc.va.gov/VISN16/docs/UMHS_Handbook_1160.pdf [accessed online 07/12/2013].
Kwiatkowska, M. and Kielan, K., 2013, "Fuzzy logic and semiotic methods in modeling of medical concepts", Fuzzy Sets and Systems, 214(1): 35-50.
Langlois, J.A., Rutland-Brown, W., and Thomas, K.E., 2006, Traumatic brain injury in the United States: Emergency department visits, hospitalizations, and deaths, Atlanta, GA: Centers for Disease Control and Preventation, National Center for Injury Preventation and Control.
Lee, S.J. and Zelen, M., 1998, "Scheduling periodic examinations for the early detection of disease: Applications to breast cancer", Journal of the American Statistical Association, 93(444): 1271-1281.
Lee, W., Nagubadi, S., Kryger, M.H., and Mokhlesi, B., 2008, "Epidemiology of obstructive sleep apnea: A population-based perspective", Expert Review of Respiratory Medicine, 2(3): 349-364.
Bibliography
152
Lewis, V., Creamer, M., and Failla, S., 2009, "Is poor sleep in veterans a function of post-traumatic stress disorder?", Military Medicine, 174(9): 948-951.
Liao, C.C., Chiu, W.T., Yeh, C.C., Chang, H.C., and Chen, T.L., 2012, "Risk and outcomes for traumatic brain injury in patients with mental disorders", Journal of Neurology, Neurosurgery, and Psychiatry, 83(12): 1186-1192.
Lin, R.H. and Chuang, C.L., 2010, "A hybrid diagnosis model for determining the types of the liver disease", Computers in Biology and Medicine, 40(7): 665-670.
Ling, G., Bandak, F., Armonda, R., Grant, G., and Ecklund, J., 2009, "Explosive blast neurotrauma", Journal Neurotrauma, 26(6): 815-825.
Lisboa, P.J. and Taktak, A.F.G., 2006, "The use of artificial neural networks in decision support in cancer: A systematic review", Neural Networks, 19(4): 408-415.
Liu, C.-F., Manning, W.G., Burgess, J.F.J., Hebert, P.L., Bryson, C.L., Fortney, J., Perkins, M., Sharp, N.D., and Maciejewski, M.L., 2011, "Reliance on Veterans Affairs outpatient care by Medicare-eligible veterans", Medical Care, 49(10): 911-917.
Lloberes, P., Duran-Cantolla, J., Martinez-Garcia, M.A., Marin, J.M., Ferrer, A., Corral, J., Masa, J.F., Parra, O., Alonso-Alvarez, M.L., and Teran-Santos, J., 2011, "Diagnosis and treatment of sleep apnea-hypopnea syndrome", Archivos De Bronconeumologia, 47(3): 143-156.
Lootsma, F.A., 1997, Fuzzy logic for planning and decision making, Dordrecht, The Netherlands: Kluwer Academic Publishers.
Maas, A.I.R., Stocchetti, N., and Bullock, R., 2008, "Moderate and severe traumatic brain injury in adults", Lancet Neurology, 7(8): 728-741.
Madani, M., Madani, F., and Akbari, M., 2013, Morbidity and mortality of obstructive sleep apnea, in Encyclopedia of Sleep, K. Clete, Editor, Waltham, MA: Academic Press, p. 339-345.
Magruder, K.M., Frueh, B.C., Knapp, R.G., Davis, L., Hamner, M.B., Martin, R.H., Gold, P.B., and Arana, G.W., 2005, "Prevalence of posttraumatic stress disorder in Veterans Affairs primary care clinics", General Hospital Psychiatry, 27(3): 169-179.
Maillart, L.M., Ivy, J.S., Ransom, S., and Diehl, K., 2008, "Assessing dynamic breast cancer screening policies", Operations Research, 56(6): 1411-1427.
Malone, F.D., Canick, J.A., Ball, R.H., Nyberg, D.A., Comstock, C.H., Bukowski, R., Berkowitz, R.L., Gross, S.J., Dugoff, L., Craigo, S.D., Timor-Tritsch, I.E., Carr, S.R., Wolfe, H.M., Dukes, K., Bianchi, D.W., Rudnicka, A.R., Hackshaw, A.K., Lambert-Messerlian, G., Wald, N.J., and D'Alton, M.E., 2005, "First-trimester or second-trimester screening, or both, for Down's Syndrome", New England Journal of Medicine, 353(19): 2001-2011.
Bibliography
153
Manderscheid, R., Druss, B., and Freeman, E., 2008, "Data to manage the mortality crisis", International Journal of Mental Health, 37(2): 49-68.
Mangiameli, P., West, D., and Rampal, R., 2004, "Model selection for medical diagnosis decision support systems", Decision Support Systems, 36(3): 247-259.
Mannarino, M.R., Di Filippo, F., and Pirro, M., 2012, "Obstructive sleep apnea syndrome", European Journal of Internal Medicine, 23(7): 586-593.
Marble, R.P. and Healy, J.C., 1999, "A neural network approach to the diagnosis of morbidity outcomes in trauma care", Artificial Intelligence in Medicine, 15(3): 299-307.
Martin, E.M., Lu, W.C., Helmick, K., French, L., and Warden, D.L., 2008, "Traumatic brain injuries sustained in the Afghanistan and Iraq wars", American Journal of Nursing, 108(4): 40-47.
Maruta, J., Lee, S.W., Jacobs, E.F., and Ghajar, J., 2010, "A unified science of concussion", Annals of the New York Academy of Sciences, 1208: 58-66.
Matis, G. and Birbilis, T., 2008, "The Glasgow Coma Scale--a brief review: Past, present, future", Acta Neurologica Belgica, 108(3): 75-89.
Matthews, E.E. and Aloia, M.S., 2009, "Continuous positive airway pressure treatment and adherence in obstructive sleep apnea", Sleep Medicine Clinics, 4(4): 473-485.
Mayo Clinic, 2012, Sleep apnea treatments and drugs, Diseases and Conditions, http://www.mayoclinic.com/health/sleep-apnea/DS00148/DSECTION=treatments-and-drugs [accessed online 08/09/2013].
McCrea, M.A., 2008, Mild traumatic brain injury and postconcussion syndrome: The new evidence base for diagnosis and treatment, New York, NY: Oxford University Press, Inc.
McLay, R.N., Klam, W.P., and Volkert, S.L., 2010, "Insomnia is the most commonly reported symptom and predicts other symptoms of post-traumatic stress disorder in U.S. service members returning from military deployments", Military Medicine, 175(10): 759-762.
Medjahed, H., Istrate, D., Boudy, J., Baldinger, J.L., Bougueroua, L., Dhouib, M.A., and Dorizzi, B., 2012, A fuzzy logic approach for remote healthcare monitoring by learning and recognizing human activities of daily living, in Fuzzy Logic - Emerging Technologies and Applications, E.P. Dadios, Editor, New York, NY: InTech.
Mehrez, A., Sinuany-Stern, Z., Arad-Geva, T., and Binyamin, S., 1996, "On the implementation of quantitative facility location models: The case of a hospital in a rural region", Journal of the Operational Research Society, 47(5): 612-625.
Melachrinoudis, E. and Min, H., 2000, "The dynamic relocation and phase-out of a hybrid, two-echelon plant/warehousing facility: A multiple objective approach", European Journal of Operational Research, 123(1): 1-15.
Bibliography
154
Middleton, P.M., 2012, "Practical use of the Glasgow Coma Scale: A comprehensive narrative review of GCS methodology", Australasian Emergency Nursing Journal, 15(3): 170-183.
Miller, W.R., Sorensen, J.L., Selzer, J.A., and Brigham, G.S., 2006, "Disseminating evidence-based practices in substance abuse treatment: A review with suggestions", Journal of Substance Abuse Treatment, 31(1): 25-39.
Milliken, C.S., Auchterlonie, J.L., and Hoge, C.W., 2007, "Longitudinal assessment of mental health problems among active and reserve component soldiers returning from the Iraq war", Journal of the American Medical Association (JAMA), 298(18): 2141-2148.
Mohr, D.C., Siddique, J., Ho, J., Duffecy, J., Jin, L., and Fokuo, J.K., 2010, "Interest in behavioral and psychological treatments delivered face-to-face, by telephone, and by internet", Annals of Behavioral Medicine, 40(1): 89-98.
Morgenthaler, T.I., Kagramanov, V., Hanak, V., and Decker, P.A., 2006, "Complex sleep apnea syndrome: Is it a unique clinical syndrome?", Sleep, 29(9): 1203-1209.
Murdoch, M., van Ryn, M., Hodges, J., and Cowper, D., 2005, "Mitigating effect of Department of Veterans Affairs disability benefits for post-traumatic stress disorder on low income", Military Medicine, 170(2): 137-140.
Murray, A.T., 2001, "Strategic analysis of public transport coverage", Socio-Economic Planning Sciences, 35(3): 175-188.
Mysliwiec, V., McGraw, L., Pierce, R., Smith, P., Trapp, B., and Roth, B.J., 2013, "Sleep disorders and associated medical comorbidities in active duty military personnel", Sleep, 36(2): 167-174.
Nathanson, B.H., Higgins, T.L., Giglio, R.J., Munshi, I.A., and Steingrub, J.S., 2003, "An exploratory study using data envelopment analysis to assess neurotrauma patients in the intensive care unit", Health Care Management Science, 6(1): 43-55.
National Academy of Engineering and Institute of Medicine, 2005, Building a better delivery system: A new engineering/healthcare partnership, Washington, DC: The National Academies Press.
National Comorbidity Survey-Replication (NCS-R), 2007, Lifetime prevalence of DSM-IV/WMH-CIDI disorders by sex and cohort, NCS-R Lifetime Prevalence Estimates, http://www.hcp.med.harvard.edu/ncs/ftpdir/NCS-R_Lifetime_Prevalence_Estimates.pdf [accessed online 08/04/2013].
National Institute of Neurological Disorders and Stroke, National Institutes of Health, 2013, Traumatic brain injury: Hope through research, NINDS Traumatic Brain Injury Information Page, http://www.ninds.nih.gov/disorders/tbi/detail_tbi.htm [accessed online 06/30/2013].
Ndiaye, M. and Alfares, H., 2008, "Modeling health care facility location for moving population groups", Computers & Operations Research, 35(7): 2154-2161.
Bibliography
155
Norquist, G.S. and Regier, D.A., 1996, "The epidemiology of psychiatric disorders and the de facto mental health care system", Annual Review of Medicine, 47: 473-479.
Norris, C.M., Ghali, W.A., Saunders, L.D., Brant, R., Galbraith, D., Faris, P., and Knudtson, M.L., 2006, "Ordinal regression model and the linear regression model were superior to the logistic regression models", Journal of Clinical Epidemiology, 59(5): 448-456.
Norris, F.H. and Hamblen, J.L., 2004, Standardized self-report measures of civilian trauma and PTSD, in Assessing Psychological Trauma and PTSD, J.P. Wilson and T.M. Keane, Editors, New York, NY: Guilford Press.
Okie, S., 2005, "Traumatic brain injury in the war zone", New England Journal of Medicine, 352(20): 2043-2047.
Orsillo, S.M., Weathers, F.W., Litz, B.T., Steinberg, H.R., Huska, J.A., and Keane, T.M., 1996, "Current and lifetime psychiatric disorders among veterans with war zone-related posttraumatic stress disorder", Journal of Nervous and Mental Disease, 184(5): 307-313.
Owen, S.H. and Daskin, M.S., 1998, "Strategic facility location: A review", European Journal of Operational Research, 111(3): 423-447.
Ozekici, S. and Pliska, S.R., 1991, "Optimal scheduling of inspections: A delayed Markov model with false positives and negatives", Operations Research, 39(2): 261-273.
Peng, C.-Y.J. and Nichols, R.N., 2003, "Using multinomial logistic models to predict adolescent behavioral risk", Journal of Modern Applied Statistical Methods, 2(1): 177-188.
Prigerson, H.G., Maciejewski, P.K., and Rosenheck, R.A., 2001, "Combat trauma: Trauma with highest risk of delayed onset and unresolved posttraumatic stress disorder symptoms, unemployment, and abuse among men", Journal of Nervous and Mental Disease, 189(2): 99-108.
Prins, A., Ouimette, P., Kimerling, R., Camerond, R.P., Hugelshofer, D.S., Shaw-Hegwer, J., Thrailkill, A., Gusman, F.D., and Sheikh, J.I., 2003, "The primary Care PTSD screen (PC-PTSD): Development and operating characteristics", Primary Care Psychiatry, 9(1): 9-14.
Punjabi, N.M., Welch, D., and Strohl, K., 2000, "Sleep disorders in regional sleep centers: A national cooperative study", Sleep, 23(4): 471-480.
Punjabi, N.M., 2008, "The epidemiology of adult obstructive sleep apnea", Proceedings of the American Thoracic Society, 5(2): 136-143.
Ramchand, R., Schell, T.L., Karney, B.R., Osilla, K.C., Burns, R.M., and Caldarone, L.B., 2010, "Disparate prevalence estimates of PTSD among service members who served in Iraq and Afghanistan: Possible explanations", Journal of Traumatic Stress, 23(1): 59-68.
Rao, V. and Lyketsos, C., 2000, "Neuropsychiatric sequelae of traumatic brain injury", Psychosomatics, 41(2): 95-103.
Bibliography
156
Regan, T., 2004, High survival rate for US troops wounded in Iraq, Christian Science Monitor.
Remzi, M. and Djavan, B., 2004, "Artificial neural networks in urology 2004", European Urology Supplements, 3(3): 33-38.
ReVelle, C.S. and Swain, R.W., 1970, "Central facilities location", Geographical Analysis, 2(1): 30-42.
Rosenfeld, J.V. and Ford, N.L., 2010, "Bomb blast, mild traumatic brain injury and psychiatric morbidity: A review", Injury, 41(5): 437-443.
Rosenheck, R.A. and Fontana, A.F., 2007, "Recent trends in VA treatment of post-traumatic stress disorder and other mental disorders", Health Affairs, 26(6): 1720-1727.
Ross, T.J., 1995, Fuzzy logic with engineering applications, New York, NY: McGraw-Hill, Inc.
Ryan, S., Kent, B.D., and McNicholas, W.T., 2011, "Genetics of cardiovascular consequences of obstructive sleep apnea syndrome", Sleep Medicine Clinics, 6(2): 247-256.
Santibanez, P., Bekiou, G., and Yip, K., 2009, "Fraser Health uses mathematical programming to plan its inpatient hospital network", Interfaces, 39(3): 196-208.
Sasaki, S., Comber, A., Suzuki, H., and Brunsdon, C., 2010, "Using genetic algorithms to optimise current and future health planning - The example of ambulance locations", International Journal of Health Geographics, 9(4): 1-10.
Schennach, R., Musil, R., Moller, H.J., and Riedel, M., 2012, "Functional outcomes in schizophrenia: Employment status as a metric of treatment outcome", Current Psychiatry Reports, 14(3): 229-236.
Schmidt-Nowara, W., 2012, Sleep apnea in adults (Beyond the basics), UpToDate, http://www.uptodate.com/contents/sleep-apnea-in-adults-beyond-the-basics?source=related_link [accessed online 08/09/2013].
Schnurr, P.P. and Green, B.L., 2004, "Understanding relationships among trauma, post-tramatic stress disorder, and health outcomes", Advances in Mind-Body Medicine, 20(1): 18-29.
Schnurr, P.P., Hayes, A.F., Lunney, C.A., McFall, M., and Uddo, M., 2006, "Longitudinal analysis of the relationship between symptoms and quality of life in veterans treated for posttraumatic stress disorder", Journal of Consulting and Clinical Psychology, 74(4): 707-713.
Schnurr, P.P. and Lunney, C.A., 2011, "Work-related quality of life and posttraumatic stress disorder symptoms among female veterans", Womens Health Issues, 21(4 Suppl): 169-175.
Schnurr, P.P. and Lunney, C.A., 2012, "Work-related outcomes among female veterans and service members after treatment of posttraumatic stress disorder", Psychiatric Services, 63(11): 1072-1079.
Bibliography
157
Schnurr, P.P., Friedman, M.J., Oxman, T.E., Dietrich, A.J., Smith, M.W., Shiner, B., Forshay, E., Gui, J., and Thurston, V., 2013, "RESPECT-PTSD: Re-engineering systems for the primary care treatment of PTSD, a randomized controlled trial", Journal of General Internal Medicine, 28(1): 32-40.
Seal, K.H., Bertenthal, D., Miner, C.R., Sen, S., and Marmar, C., 2007, "Bringing the war back home: Mental health disorders among 103,788 US veterans returning from Iraq and Afghanistan seen at Department of Veterans Affairs facilities", Archives of Internal Medicine, 167(5): 476-482.
Seal, K.H., Bertenthal, D., Maguen, S., Gima, K., Chu, A., and Marmar, C.R., 2008, "Getting beyond "don't ask; don't tell": An evaluation of US Veterans Administration postdeployment mental health screening of veterans returning from Iraq and Afghanistan", American Journal of Public Health, 98(4): 714-20.
Seelig, A.D., Jacobson, I.G., Smith, B., Hooper, T.I., Boyko, E.J., Gackstetter, G.D., Gehrman, P., Macera, C.A., and Smith, T.C., 2010, "Sleep patterns before, during, and after deployment to Iraq and Afghanistan", Sleep, 33(12): 1615-1622.
Sentas, P., Angelis, L., Stamelos, I., and Bleris, G., 2005, "Software productivity and effort prediction with ordinal regression", Information and Software Technology, 47(1): 17-29.
Sentas, P. and Angelis, L., 2006, "Categorical missing data imputation for software cost estimation by multinomial logistic regression", Journal of Systems and Software, 79(3): 404-414.
Shalev, A.Y., Freedman, S., Peri, T., Brandes, D., Sahar, T., Orr, S.P., and Pitman, R.K., 1998, "Prospective study of posttraumatic stress disorder and depression following trauma", American Journal of Psychiatry, 155(5): 630-637.
Shalev, A.Y., 2009, "Posttraumatic stress disorder (PTSD) and stress related disorders", Psychiatric Clinics of North America, 32(3): 687-704.
Sharafkhaneh, A., Richardson, P., and Hirshkowitz, M., 2004, "Sleep apnea in a high risk population: A study of Veterans Health Administration beneficiaries", Sleep Medicine, 5(4): 345-350.
Sharafkhaneh, A., Giray, N., Richardson, P., Young, T., and Hirshkowitz, M., 2005, "Association of psychiatric disorders and sleep apnea in a large cohort", Sleep, 28(11): 1405-1411.
Shenton, M.E., Hamoda, H.M., Schneiderman, J.S., Bouix, S., Pasternak, O., Rathi, Y., Vu, M.A., Purohit, M.P., Helmer, K., Koerte, I., Lin, A.P., Westin, C.F., Kikinis, R., Kubicki, M., Stern, R.A., and Zafonte, R., 2012, "A review of magnetic resonance imaging and diffusion tensor imaging findings in mild traumatic brain injury", Brain Imaging and Behavior, 6(2): 137-192.
Bibliography
158
Shiner, B., D'Avolio, L.W., Nguyen, T.M., Zayed, M.H., Young-Xu, Y., Desai, R.A., Schnurr, P.P., Fiore, L.D., and Watts, B.V., 2012, "Measuring use of evidence based psychotherapy for posttraumatic stress disorder", Administration and Policy in Mental Health and Mental Health Services Research, Epub ahead of print.
Shiner, B., Drake, R.E., Watts, B.V., Desai, R.A., and Schnurr, P.P., 2012, "Access to VA Services for Returning Veterans with PTSD", Military Medicine, 177(7): 814-822.
Sin, D.D., Fitzgerald, F., Parker, J.D., Newton, G., Floras, J.S., and Bradley, T.D., 1999, "Risk factors for central and obstructive sleep apnea in 450 men and women with congestive heart failure", American Journal of Respiratory and Critical Care Medicine, 160(4): 1101-1106.
Smith, B.M., Evans, C.T., Hines E., 2013, Mild traumatic brain injury: Screening and comorbidities, Forum, Health Services Research & Development, http://www.hsrd.research.va.gov/publications/forum/aug13/aug13-4.cfm#.UgBTRdKGt8E [accessed online 08/05/2013].
Smith, M.W., Schnurr, P.P., and Rosenheck, R.A., 2005, "Employment outcomes and PTSD symptom severity", Mental Health Services Research, 7(2): 89-101.
Sorensen, J.L., 2011, "From Cat's Cradle to Beat the Reaper: Getting evidence-based treatments into practice in spite of ourselves", Addictive Behaviors, 36(6): 597-600.
Sut, N. and Senocak, M., 2007, "Assessment of the performances of multilayer perceptron neural networks in comparison with recurrent neural networks and two statistical methods for diag-nosing coronary artery disease", Expert Systems, 24(3): 131-147.
Syam, S.S. and Côté, M.J., 2010, "A location-allocation model for service providers with application to not-for-profit health care organizations", Omega, 38(3-4): 157-166.
Tanielian, T. and Jaycox, L.H., 2008, Invisible wounds of war: Psychological and cognitive injuries, their consequences, and services to assist recovery, Santa Monica, CA: RAND, Center for Military Health Policy Research.
Teasdale, G. and Jennett, B., 1974, "Assessment of coma and impaired consciousness: A practical scale", The Lancet, 304(7872): 81-84.
Thatcher, R.W., 2000, "EEG operant conditioning (biofeedback) and traumatic brain injury", Clinical Electroencephalography, 31(1): 38-44.
Thatcher, R.W., North, D.M., Curtin, R.T., Walker, R.A., Biver, C.J., Gomez, J.F., and Salazar, A.M., 2001, "An EEG severity index of traumatic brain injury", Journal of Neuropsychiatry and Clinical Neurosciences, 13(1): 77-87.
Thibault, J.M. and Steiner, R.W., 2004, "Efficient identification of adults with depression and dementia", American Family Physician, 70(6): 1101-1110.
Bibliography
159
Toregas, C., 1970, A covering formulation for the location of public service facilities, M.S. thesis, Cornell University,
Torshabi, A.E., Riboldi, M., Pella, A., Negarestani, A., Rahnema, M., and Baroni, G., 2012, A clinical application of fuzzy logic, in Fuzzy Logic - Emerging Technologies and Applications, E.P. Dadios, Editor, New York, NY: InTech.
Tutty, S., Spangler, D.L., Poppleton, L.E., Ludman, E.J., and Simon, G.E., 2010, "Evaluating the Effectiveness of Cognitive-Behavioral Teletherapy in Depressed Adults", Behavior Therapy, 41(2): 229-236.
U.S. Department of Health and Human Services, Health Resources and Services Administration (HRSA), 2006, Traumatic brain injury: An introduction, https://tbitac.hrsa.gov/download/ScreeningInstruments508.pdf [accessed online 07/01/2013].
U.S. Department of Health and Human Services, 2012, What is sleep apnea?, NHLBI: Health Information for the Public, http://www.nhlbi.nih.gov/health/health-topics/topics/sleepapnea [accessed online 07/07/2013].
U.S. Department of Health and Human Services, National Institutes of Health, National Institute of Mental Health, 2013, Post-traumatic stress disorder (PTSD), http://www.nimh.nih.gov/health/publications/post-traumatic-stress-disorder-ptsd/nimh_ptsd_booklet.pdf [accessed online 08/04/2013].
U.S. Department of Veterans Affairs (VA), 2012, PTSD (ptsd51), VA Measure Master, http://vaww.reporting.oqp.med.va.gov [accessed online 11/02/2012].
U.S. Department of Veterans Affairs (VA), 2013, VISN 1: VA New England Healthcare System, http://www.newengland.va.gov [accessed online 07/12/2013].
U.S. Department of Veterans Affairs (VA) and Department of Defense (DoD), 2010, VA/DOD clinical practice guideline for management of posttraumatic stress, http://www.healthquality.va.gov/ptsd/cpg_PTSD-FULL-201011612.pdf [accessed online 01/14/2013].
Upadhyaya, S., Farahmand, K., and Baker-Demaray, T., 2013, "Comparison of NN and LR classifiers in the context of screening native American elders with diabetes", Expert Systems with Applications, 40(15): 5830-5838.
U.S. Department of Veterans Affairs, National Center for PTSD, 2013, Treatment of PTSD, Information on Trauma and PTSD, http://www.ptsd.va.gov/public/pages/treatment-ptsd.asp [accessed online 08/01/2013].
U.S. Department of Veterans Affairs, Office of Public Health, 2011a, Analysis of VA health care utilization among Operation Enduring Freedom (OEF), Operation Iraqi Freedom (OIF), and Operation New Dawn (OND) Veterans: Cumulative from 1st Qtr FY 2002 through 4th Qtr FY 2011 (October 1, 2001 – September 30, 2011), Reports on OEF/OIF/OND Veterans,
Bibliography
160
http://www.publichealth.va.gov/docs/epidemiology/healthcare-utilization-report-fy2011-qtr4.pdf [accessed online 07/11/2013].
U.S. Department of Veterans Affairs, Office of Public Health, 2011b, Report on VA facility specific Operation Enduring Freedom (OEF), Operation Iraqi Freedom (OIF), and Operation New Dawn (OND) veterans coded with potential PTSD: Cumulative from 1st Qtr FY 2002 through 4th Qtr FY 2011 (October 1, 2001 – September 30, 2011), Reports on OEF/OIF/OND Veterans, http://www.publichealth.va.gov/docs/epidemiology/ptsd-report-fy2011-qtr4.pdf [accessed online 07/11/2013].
Vaishnavi, S., Rao, V., and Fann, J.R., 2009, "Neuropsychiatric problems after Traumatic Brain Injury: Unraveling the silent epidemic", Psychosomatics, 50(3): 198-205.
van Dam, D., Ehring, T., Vedel, E., and Emmelkamp, P.M.G., 2010, "Validation of the Primary Care Posttraumatic Stress Disorder screening questionnaire (PC-PTSD) in civilian substance use disorder patients", Journal of Substance Abuse Treatment, 39(2): 105-113.
Vaughn, M.L., Cavill, S.J., Taylor, S.J., Foy, M.A., and Fogg, A.J.B., 2000, Direct knowledge discovery and interpretation from a Multilayer Perceptron network which performs low-back-pain classification, in Hybrid Neural Systems, S. Wermter and R. Sun, Editors, New York, NY: Springer.
Veryha, Y. and Adamczyk, K., 2005, "Application of fuzzy classification in modern primary dental care", Informatics in Primary Care, 13(1): 23-34.
Veterans Service Support Center, 2011a, Outpatient cube, Decision Support System Portal, https://vssc.med.va.gov/dss_reports/cubeOutpatient.asp [accessed online 06/25/2011].
Veterans Service Support Center, 2011b, VA outpatient service use in fiscal year 2010, ProClarity Outpatient NCPD Cube, http://klfmenu.med.va.gov [accessed online 06/21/2011].
Vozoris, N.T., 2012, "Sleep apnea-plus: Prevalence, risk factors, and association with cardiovascular diseases using United States population-level data", Sleep Medicine, 13(6): 637-644.
Wang, P.S., Berglund, P., Olfson, M., Pincus, H.A., Wells, K.B., and Kessler, R.C., 2005, "Failure and delay in initial treatment contact after first onset of mental disorders in the National Comorbidity Survey Replication", Archives of General Psychiatry, 62(6): 603-613.
Wang, Y., 2005, "A multinomial logistic regression modeling approach for anomaly intrusion detection", Computers & Security, 24(8): 662-674.
Warden, D., 2006, "Military TBI during the Iraq and Afghanistan wars", Journal of Head Trauma Rehabilitation, 21(5): 398-402.
Bibliography
161
Warwick, J. and Duffy, S.W., 2005, "A review of cancer screening evaluation techniques, with some particular examples in breast cancer screening", Journal of the Royal Statistical Society, Series A (Statistics in Society), 168(4): 657-677.
Watts, B.V., Schnurr, P.P., Zayed, M., and Sinnott, P.L., 2012, "Effects of a patient decision aid for veterans with posttraumatic stress disorder (PTSD)", in VA Health Services Research and Development Service (HSR&D) and Quality Enhancement Research Initiative (QUERI) National Conference, National Harbor, MD.
Weathers, F.W., Litz, B.T., Herman, D.S., Huska, J.A., and Keane, T.M., 1993, "The PTSD Checklist (PCL): Reliability, validity, and diagnostic utility", in Annual Meeting of International Society for Traumatic Stress Studies, San Antonio, TX.
Weathers, F.W., Ruscio, A.M., and Keane, T.M., 1999, "Psychometric properties of nine scoring rules for the clinician-administered posttraumatic stress disorder scale", Psychological Assessment, 11(2): 124-133.
Weathers, F.W., Keane, T.M., and Davidson, J.R.T., 2001, "Clinician-administered PTSD scale: A review of the first ten years of research", Depression and Anxiety, 13: 132-156.
West, D. and West, V., 2000, "Model selection for a medical diagnostic decision support system: A breast cancer detection case", Artificial Intelligence in Medicine, 20(3): 183-204.
West, D., Mangiameli, P., Rampal, R., and West, V., 2005, "Ensemble strategies for a medical diagnostic decision support system: A breast cancer diagnosis application", European Journal of Operational Research, 162(2): 532-551.
WHO International Consortium in Psychiatric Epidemiology, 2000, "Cross-national comparisons of the prevalences and correlates of mental disorders", Bulletin of the World Health Organization, 78(4): 413-426.
Wijdicks, E.F., Bamlet, W.R., Maramattom, B.V., Manno, E.M., and McClelland, R.L., 2005, "Validation of a new coma scale: The FOUR score", Annals of Neurology, 58(4): 585-593.
Wilson, J.A. and Wells, M.G., 2009, "Telehealth and the deaf: A comparison study", Journal of Deaf Studies and Deaf Education, 14(3): 386-402.
Wojcik, B.E., Stein, C.R., Bagg, K., Humphrey, R.J., and Orosco, J., 2010, "Traumatic brain injury hospitalizations of US army soldiers deployed to Afghanistan and Iraq", American Journal of Preventive Medicine, 38(1): S108-S116.
Wu, X., Hu, J., Zhuo, L., Fu, C., Hui, G., Wang, Y., Yang, W., Teng, L., Lu, S., and Xu, G., 2008, "Epidemiology of traumatic brain injury in Eastern China, 2004: A prospective large case study", Journal of Trauma, 64(5): 1313-1319.
Yaggi, H.K. and Strohl, K.P., 2010, "Adult obstructive sleep apnea/hypopnea syndrome: Definitions, risk factors, and pathogenesis", Clinics in Chest Medicine, 31(2): 179-186.
Bibliography
162
Young, T., Palta, M., Dempsey, J., Skatrud, J., Weber, S., and Badr, S., 1993, "The occurrence of sleep-disordered breathing among middle-aged adults", New England Journal of Medicine, 328(17): 1230-1235.
Young, T., Skatrud, J., and Peppard, P.E., 2004, "Risk factors for obstructive sleep apnea in adults", Journal of the American Medical Association (JAMA), 291(16): 2013-2016.
Yu, W., Ravelo, A., Wagner, T.H., Phibbs, C.S., Bhandari, A., Chen, S., and Barnett, P.G., 2003, "Prevalence and costs of chronic conditions in the VA health care system", Medical Care Research and Review, 60(3 Suppl): 146S-167S.
Zadeh, L.A., 1965, "Fuzzy sets", Information and Control, 8(3): 338-353.
Zhu, C.W., Livote, E.E., Ross, J.S., and Penrod, J.D., 2010, "A random effects multinomial logit analysis of using Medicare and VA healthcare among veterans with dementia", Home Health Care Services Quarterly, 29(2): 91-104.
Zuercher, M., Ummenhofer, W., Baltussen, A., and Walder, B., 2009, "The use of Glasgow Coma Scale in injury assessment: a critical review", Brain Injury, 23(5): 371-384.
Appendix A - Overall Sensitivity of PTSD Screening Process
163
Appendix A - Overall Sensitivity of PTSD Screening Process
In order to introduce some notation and motivate the general approach, the one-year analysis
horizon is considered first. In this process, a truly positive patient can be given a final correct
positive diagnosis in the following two mutually exclusive and collectively exhaustive manners.
Note that PC-PTSD, PCL, and CAPS tests are represented by S, P, and C, respectively.
(S, S+, P, P?� , P+): – Patient takes PC-PTSD,
– PC-PTSD correctly identifies patient as positive,
– Patient is sent to PCL for review and takes it, and
– PCL makes a definite diagnosis and confirms patient as positive.
(S, S+, P, P?, C, C+): – Patient takes PC-PTSD,
– PC-PTSD correctly identifies patient as positive,
– Patient is sent to PCL for review and takes it,
– PCL cannot make a definite diagnosis,
– Patient is sent to CAPS for further review and takes it, and
– CAPS confirms patient as positive.
The probability of detecting a truly positive patient in one year, p1, then is the sum of the
probabilities of these events:
Appendix A - Overall Sensitivity of PTSD Screening Process
164
p1 = Probability of detecting a truly positive patient in one year
= P ��S, S+, P, P?� , P+� or �S, S+, P, P?, C, C+��
= P�S�P�S+�P�P��1− P�P?��P{P+} + P�S�P�S+�P�P�P�P?�P�C�P�C+�
= rs(1 − αs)rp(1 − u)�1− αp� + rs(1 − αs)rpurc(1− αc)
= rs(1 − αs) �rp(1− u)�1 − αp� + rpurc(1− αc)�
= rs(1 − αs)rp �1 − αp − u �1 − αp − rc(1 − αc)��. (A-1)
More generally, a correct positive determination in a j-year horizon can occur in the following
manners:
((((S1, S1
-) or S1����� or (S1
, S1+, P����) or (S1
, S1+, P, P?, C����)), …, ((Si-1
, Si-1-) or Si-1
������ or
(Si-1, Si-1
+, P����) or (Si-1, Si-1
+, P, P?, C����)), Si, Si
+, P, P?� , P+), i ≤ j):
− PC-PTSD in year i, 1 ≤ i ≤ j, correctly identifies patient as positive after i-1 ≥ 0 years being determined as negative, skipping PC-PTSD, or being determined as positive by PC-PTSD but not following up with confirmatory testing,
− Patient is sent to PCL for review and takes it, and
− PCL makes a definite diagnosis and confirms patient as positive.
((((S1, S1
-) or S1����� or (S1
, S1+, P����) or (S1
, S1+, P, P?, C����)), …, ((Si-1
, Si-1-) or Si-1
������ or
(Si-1, Si-1
+, P����) or (Si-1, Si-1
+, P, P?, C����)), Si, Si
+, P, P?, C, C+), i ≤ j):
− PC-PTSD in year i, 1 ≤ i ≤ j, correctly identifies patient as positive after i-1 ≥ 0 years being determined as negative or not skipping PC-PTSD, or being determined as positive by PC-PTSD but not following up with confirmatory testing,
− Patient is sent to PCL for review and takes it,
− PCL cannot make a definite diagnosis,
− Patient is sent to CAPS for further review and takes it, and
− CAPS confirms patient as positive.
Appendix A - Overall Sensitivity of PTSD Screening Process
165
Note that both events above can occur in j manners, as the patient can be identified as positive by
either PC-PTSD test in year i = 1, 2, …, or j. Now, the generalized analog to Eq. (A-1) is
pj = Probability of detecting a truly positive patient in j years
= P ����S1, S1
−� or S1����� or �S1
, S1+, P����� or �S1
, S1+, P, P?, C������ , … ,
��Si−1, Si−1
−� or Si−1������� or �Si−1
, Si−1+, P����� or �Si−1
, Si−1+, P, P?, C������,
Si, Si
+, P, P?� , P+� , i ≤ j�
+P ����S1, S1
−� or S1����� or �S1
, S1+, P����� or �S1
, S1+, P, P?, C������ , … ,
��Si−1, Si−1
−� or Si−1������� or �Si−1
, Si−1+, P����� or �Si−1
, Si−1+, P, P?, C������,
Si, Si
+, P, P?, C, C+� , i ≤ j�
= P ����S1, S1
−� or S1����� or �S1
, S1+, P����� or �S1
, S1+, P, P?, C������ , … ,
��Si−1, Si−1
−� or Si−1������� or �Si−1
, Si−1+, P����� or �Si−1
, Si−1+, P, P?, C������,
Si, Si
+� , i ≤ j� × �P �P, P?� , P+� + P�P, P?, C, C+��
= P ����S1, S1
−� or S1����� or �S1
, S1+, P����� or �S1
, S1+, P, P?, C������ , … ,
��Si−1, Si−1
−� or Si−1������� or �Si−1
, Si−1+, P����� or �Si−1
, Si−1+, P, P?, C������,
Si, Si
+� , i ≤ j� × P�P� × �P �P?� �P{P+} + P�P?�P�C�P�C+��
= ∑ �rsαs + 1 − rs + rs(1− αs)�1 − rp� + rs(1 − αs)rpu(1 − rc)�irs(1− αs)j−1
i=0
× rp �(1 − u)�1 − αp� + urc(1− αc)�
= ∑ �1 − rs(1− αs)rp�1 − u(1 − rc)��irs(1− αs)j−1
i=0 rp �1 − αp − u �1 − αp − rc(1− αc)��.
(A-2)
Appendix A - Overall Sensitivity of PTSD Screening Process
166
Using the finite series
∑ axik−1i=0 = a 1−xk
1−x, (A-3)
Eq. (A-2) becomes
pj =1−�1−rs(1−αs)rp�1−u(1−rc)��
j
1−�1−rs(1−αs)rp�1−u(1−rc)��rs(1− αs)rp �1 − αp − u �1 − αp − rc(1− αc)��
=1−�1−rs(1−αs)rp�1−u(1−rc)��
j
rs(1−αs)rp�1−u(1−rc)�rs(1 − αs)rp �1 − αp − u �1 − αp − rc(1 − αc)��
=�1−�1−rs(1−αs)rp�1−u(1−rc)��
j��1−αp−u�1−αp−rc(1−αc)��
1−u(1−rc) . (A-4)
Appendix B - Overall Specificity of PTSD Screening Process
167
Appendix B - Overall Specificity of PTSD Screening Process
Following the same logic as for the deviation of overall system sensitivity in Appendix A, a
correct negative determination in j years can occur in the following manners:
((S1, S1
-) or S1����� or (S1
, S1+, P����) or (S1
, S1+, P, P?, C����)), …, ((Sj
, Sj-) or Sj
���� or (Sj, Sj
+, P����)
or (Sj, Sj
+, P, P?, C����)):
− PC-PTSDs during j years either classify patient as negative, give positive result but not confirmed by confirmatory testing, or are skipped.
((((S1, S1
-) or S1����� or (S1
, S1+, P����) or (S1
, S1+, P, P?, C����)), …, ((Si-1
, Si-1-) or Si−1
������� or (Si-1
, Si-1+, P����) or (Si-1
, Si-1+, P, P?, C����)), Si
, Si+, P, P?� , P-), i ≤ j):
− PC-PTSD in year i, 1 ≤ i ≤ j, incorrectly identifies patient as positive after i-1 ≥ 0 years either being determined as negative, skipping PC-PTSD, or being determined as positive by PC-PTSD but not following up with confirmatory testing,
− Patient is sent to PCL for review and takes it, and
− PCL makes a definite diagnosis and correctly classifies patient as negative.
((((S1, S1
-) or S1����� or (S1
, S1+, P����) or (S1
, S1+, P, P?, C����)), …, ((Si-1
, Si-1-) or Si−1
������� or (Si-1
, Si-1+, P����) or (Si-1
, Si-1+, P, P?, C����)), Si
, Si+, P, P?, C, C-), i ≤ j):
− PC-PTSD in year i, 1 ≤ i ≤ j, incorrectly identifies patient as positive after i-1 ≥ 0 years being determined as negative, skipping PC-PTSD, or being determined as positive by PC-PTSD but not following up with confirmatory testing,
Appendix B - Overall Specificity of PTSD Screening Process
168
− Patient is sent to PCL for review and takes it,
− PCL cannot make a definite diagnosis,
− Patient is sent to CAPS for further review and takes it, and
− CAPS correctly classifies patient as negative.
All events above except for the first one can occur in j manners, as the patient can be identified
as positive by PC-PTSD test in either year i = 1, 2, …, or j. Now considering these events one at
a time, the probability of detecting a truly negative patient in j years is
qj = Probability of identifying a truly negative patient as negative in j years
= P ���S1, S1
−� or S1����� or �S1
, S1, P����� or �S1
, S1+, P, P?, C������ , … ,
��Sj, Sj
−� or Sj���� or �Sj
, Sj+, P����� or �Sj
, Sj+, P, P?, C�������
+P ����S1, S1
−� or S1����� or �S1
, S1, P����� or �S1
, S1+, P, P?, C������ , … ,
��Si−1, Si−1
−� or Si−1������� or �Si−1
, Si−1+, P����� or �Si−1
, Si−1+, P, P?, C������,
Si, Si
+, P, P?� , P−� , i ≤ j�
+P ����S1, S1
−� or S1����� or �S1
, S1, P����� or �S1
, S1+, P, P?, C������ , … ,
��Si−1, Si−1
−� or Si−1������� or �Si−1
, Si−1+, P����� or �Si−1
, Si−1+, P, P?, C������,
Si, Si
+, P, P?, C, C−� , i ≤ j�
Appendix B - Overall Specificity of PTSD Screening Process
169
= P ���S1, S1
−� or S1����� or �S1
, S1, P����� or �S1
, S1+, P, P?, C������ , … ,
��Sj, Sj
−� or Sj���� or �Sj
, Sj+, P����� or �Sj
, Sj+, P, P?, C�������
+P ����S1, S1
−� or S1����� or �S1
, S1, P����� or �S1
, S1+, P, P?, C������ , … ,
��Si−1, Si−1
−� or Si−1������� or �Si−1
, Si−1+, P����� or �Si−1
, Si−1+, P, P?, C������,
Si, Si
+� , i ≤ j� × �P �P, P?� , P−� + P�P, P?, C, C−��
= P ���S1, S1
−� or S1����� or �S1
, S1, P����� or �S1
, S1+, P, P?, C������ , … ,
��Sj, Sj
−� or Sj���� or �Sj
, Sj+, P����� or �Sj
, Sj+, P, P?, C�������
+P ����S1, S1
−� or S1����� or �S1
, S1, P����� or �S1
, S1+, P, P?, C������ , … ,
��Si−1, Si−1
−� or Si−1������� or �Si−1
, Si−1+, P����� or �Si−1
, Si−1+, P, P?, C������,
Si, Si
+� , i ≤ j� × P�P� × �P �P?� �P{P−} + P�P?�P�C�P{C−}�
= �rs�1− βs� + 1 − rs + rsβs�1− rp� + rsβsrpu(1− rc)�j
+∑ �rs�1 − βs� + 1 − rs + rsβs�1− rp� + rsβsrpu(1− rc)�irsβs
j−1i=0
× rp �(1 − u) �1 − βp� + urc �1 − βp��
= �1 − rsβsrp�1 − u(1 − rc)��j
+ ∑ �1 − rsβsrp�1− u(1 − rc)��irsβs
j−1i=0
× rp �1 − βp − u �1 − βp − rc�1− βc���. (B-1)
Using the finite series given in Eq. (A-3) and after some algebra, Eq. (B-1) becomes
qj = �1 − rsβsrp�1− u(1 − rc)��j
+1−�1−rsβsrp�1−u(1−rc)��
j
1−�1−rsβsrp�1−u(1−rc)��rsβsrp �1 − βp − u �1 − βp − rc�1− βc���
=�1−rsβsrp�1−u(1−rc)��
j�(1−u)βp+urcβc�+1−βp−u�1−βp−rc�1−βc��
1−u(1−rc) . (B-2)
Appendix C - Expected Total Cost of PTSD Screening Process
170
Appendix C - Expected Total Cost of PTSD Screening Process
The expected cost per n patients after j years using PC-PTSD, PCL, and CAPS tests, ECj, is
equal to the following sum:
ECj = Expected cost of all PC-PTSD tests per n patients + Expected cost of all PCL tests per n patients + Expected cost of all CAPS tests per n patients + Expected cost of all false-positives per n patients + Expected cost of all false-negatives per n patients
= �ks × Expected number of PC-PTSD tests per n patients� + �kp × Expected number of PCL tests per n patients� + (kc × Expected number of CAPS tests per n patients) + �kFP × Expected number of false-positives per n patients� + �kFN × Expected cumulative number of false-negatives per n patients�. (C-1)
For readability, it is convenient first to consider each of the above four expectations separately
and then to sum them at the end of this appendix.
Expected number of PC-PTSD tests per n patients:
The expected number of PC-PTSD tests per n patients, E[Is,n], is a sum of the expected number
of tests per single positive patient, E[Is | Positive], and the expected number of tests per single
negative patient, E[Is | Negative], conditioned as follows:
Appendix C - Expected Total Cost of PTSD Screening Process
171
E�Is, n� = n(P{Patient is positive}E[Is|Positive] + P{Patient is negative}E[Is|Negative]).
(C-2)
The expected number of PC-PTSD tests for a positive patient is
E[Is|Positive] = ∑ iP{Is = i}ji=1
= 1P{Is = 1} + 2P{Is = 2} + ⋯+ (j− 1)P{Is = j − 1} + jP{Is = j}
= 1 �∑ (1 − rs)aj−1a=0 pcon
+ + 𝑗ps_nc+ (1− rs)j−1�
+2 �∑ �(a+1)!a!1!
� (1 − rs)aps_nc+j−2
a=0 pcon+ + � j!
2!(j−2)!� �ps_nc
+ �2
(1− rs)j−2� + ⋯
+(j− 1)�∑ �(a+j−2)!a!(j−2)!
� (1 − rs)a �ps_nc+ �
j−21a=0 pcon
+ + � j!(j−1)!1!
� �ps_nc+ �
j−1(1 − rs)�
+j ��ps_nc+ �
j−1pcon+ + �ps_nc
+ �j�
= ∑ i ��ps_nc+ �
i−1pcon+ ∑ �(a+i−1)!
a!(i−1)!� (1 − rs)aj−i
a=0 + � j!i!(j−i)!
� �ps_nc+ �
i(1 − rs)j−i�j
i=1 , (C-3)
and similarly the expected number of PC-PTSD tests for a negative patient is
E[Is|Negative] = ∑ iP{Is = i}ji=1
= 1P{Is = 1} + 2P{Is = 2} + ⋯+ (j− 1)P{Is = j − 1} + jP{Is = j}
= 1 �∑ (1 − rs)aj−1a=0 pcon
− + 𝑗ps_nc− (1− rs)j−1�
+2 �∑ �(a+1)!a!1!
� (1 − rs)aps_nc−j−2
a=0 pcon− + � j!
2!(j−2)!� �ps_nc
− �2
(1− rs)j−2� + ⋯
+(j− 1)�∑ �(a+j−2)!a!(j−2)!
� (1 − rs)a �ps_nc− �
j−21a=0 pcon
− + � j!(j−1)!1!
� �ps_nc− �
j−1(1 − rs)�
+j ��ps_nc− �
j−1pcon− + �ps_nc
− �j�
= ∑ i ��ps_nc− �
i−1pcon− ∑ �(a+i−1)!
a!(i−1)!� (1 − rs)aj−i
a=0 + � j!i!(j−i)!
� �ps_nc− �
i(1 − rs)j−i�j
i=1 , (C-4)
where the probabilities pcon+ , pcon
− , ps_nc+ , and ps_nc
− are given as
Appendix C - Expected Total Cost of PTSD Screening Process
172
pcon+ = P{Result is confirmed by either PCL or CAPS|Patient is positive}
= rs(1 − αs)rp�(1− u) + urc� = rs(1− αs)rp�1− u(1 − rc)�, (C-5)
pcon− = P{Result is confirmed by either PCL or CAPS|Patient is negative}
= rsβsrp�1− u(1 − rc)�, (C-6)
ps_nc+ = P�PC-PTSD is taken but its result remains unconfirmed|Patient is positive�
= 1 − pcon+ − P�PC-PTSD is not taken� = 1 − pcon
+ − (1− rs) = rs − pcon+ , (C-7)
ps_nc− = P�PC-PTSD is taken but its result remains unconfirmed|Patient is negative�
= rs − pcon− . (C-8)
Thus, by substituting Eq. (C-3) and (C-4) into Eq. (C-2), and then by using the binomial theorem
(a + b)m = ∑ � m!y!(m−y)!
� ambm−ymy=0 , (C-9)
and after some algebra, the expected number of PC-PTSD tests per n patients is calculated as
E�Is, n� = n�p �∑ i �ps_nc+ �
i−1pcon+ ∑ �(a+i−1)!
a!(i−1)!� (1 − rs)aj−i
a=0ji=1 + 𝑗ps_nc
+ �1− pcon+ �
j−1�
+(1 − p) �∑ i �ps_nc− �
i−1pcon− ∑ �(a+i−1)!
a!(i−1)!� (1 − rs)aj−i
a=0ji=1 + 𝑗ps_nc
− �1 − pcon− �
j−1��.
(C-10)
Note that the variance in number of PC-PTSD tests per patient can be found in a similar manner,
using the relations that E�X 2� = ∑ x2P{X = x}jx=1 , Var[X] = E�X 2� − E2[X], Var[X + Y] =
Appendix C - Expected Total Cost of PTSD Screening Process
173
Var[X] + Var[Y] + 2Cov[X, Y], and Cov[X, Y] = 0 if X and Y are independent. After some
algebra, by using the same approach above, the following result for Var�Is, n� can be obtained as
Var�Is, n� = Var[Is + Is + ⋯+ Is] = Var[Is] + Var[Is] + ⋯+ Var[Is] = n(Var[Is])
= n�E�Is2� − E2[Is]�
= n�p∑ i2 ��ps_nc+ �
i−1pcon+ ∑ �(a+i−1)!
a!(i−1)!� (1 − rs)aj−i
a=0 + � j!i!(j−i)!
� �ps_nc+ �
i(1 − rs)j−i�j
i=1
+(1− p)∑ i2 ��ps_nc− �
i−1pcon− ∑ �(a+i−1)!
a!(i−1)!� (1 − rs)aj−i
a=0 + � j!i!(j−i)!
� �ps_nc− �
i(1 − rs)j−i�j
i=1
−�p �∑ i �ps_nc+ �
i−1pcon+ ∑ �(a+i−1)!
a!(i−1)!� (1 − rs)aj−i
a=0ji=1 + 𝑗ps_nc
+ �1− pcon+ �
j−1�
+(1 − p) �∑ i �ps_nc− �
i−1pcon− ∑ �(a+i−1)!
a!(i−1)!� (1− rs)aj−i
a=0ji=1 + 𝑗ps_nc
− �1− pcon− �
j−1��
2
�.
(C-11)
Expected number of PCL tests per n patients:
The expected number of PCL tests per n patients, E[Is,n], similar to the number of PC-PTSD
tests, is a sum of the expected number of tests per single positive patient, E[Ip | Positive], and the
expected number of tests per single negative patient, E[Ip | Negative], conditioned as follows:
E�Ip, n� = n�P{Patient is positive}E�Ip|Positive� + P{Patient is negative}E�Ip|Negative��.
(C-12)
The expected number of PCL tests for a positive patient is
E�Ip|Positive� = ∑ iP�Ip = i�ji=1
number of n number of n
Appendix C - Expected Total Cost of PTSD Screening Process
174
= 1P�Ip = 1� + 2P�Ip = 2� + ⋯+ (j − 1)P�Ip = j − 1� + jP�Ip = j�
= 1 �∑ �1 − rs(1 − αs)rp�aj−1
a=0 pcon+ + 𝑗pp_nc
+ �1− rs(1 − αs)rp�j−1�
+2 �∑ �(a+1)!a!1!
� �1 − rs(1− αs)rp�app_nc+j−2
a=0 pcon+ + � j!
2!(j−2)!� �pp_nc
+ �2�1 − rs(1− αs)rp�
j−2� + ⋯
+(j− 1)�∑ �(a+j−2)!a!(j−2)!
� �1 − rs(1− αs)rp�a�pp_nc
+ �j−2
1a=0 pcon
+
+ � j!(j−1)!1!
� �pp_nc+ �
j−1�1 − rs(1− αs)rp��
+j ��pp_nc+ �
j−1pcon+ + �pp_nc
+ �j�
= ∑ i ��pp_nc+ �
i−1pcon+ ∑ �(a+i−1)!
a!(i−1)!� �1 − rs(1 − αs)rp�
aj−ia=0
ji=1
+ � j!i!(j−i)!
� �pp_nc+ �
i�1 − rs(1 − αs)rp�
j−i�, (C-13)
and similarly the expected number of PCL tests for a negative patient is
E�Ip|Negative� = ∑ iP�Ip = i�ji=1
= 1P�Ip = 1� + 2P�Ip = 2� + ⋯+ (j − 1)P�Ip = j − 1� + jP�Ip = j�
= 1 �∑ �1 − rsβsrp�aj−1
a=0 pcon− + 𝑗pp_nc
− �1− rsβsrp�j−1�
+2 �∑ �(a+1)!a!1!
� �1 − rsβsrp�app_nc−j−2
a=0 pcon− + � j!
2!(j−2)!� �pp_nc
− �2�1 − rsβsrp�
j−2� + ⋯
+(j− 1)�∑ �(a+j−2)!a!(j−2)!
� �1 − rsβsrp�a�pp_nc
− �j−2
1a=0 pcon
− + � j!(j−1)!1!
� �pp_nc− �
j−1�1 − rsβsrp��
+j ��pp_nc− �
j−1pcon− + �pp_nc
− �j�
= ∑ i ��pp_nc− �
i−1pcon− ∑ �(a+i−1)!
a!(i−1)!� �1 − rsβsrp�
aj−ia=0 + � j!
i!(j−i)!� �pp_nc
− �i�1 − rsβsrp�
j−i�j
i=1 ,
(C-14)
where the probabilities pp_nc+ and pp_nc
− are given as
pp_nc+ = P{PCL is taken but its result remains unconfirmed|Patient is positive}
= rs(1− αs)rpu(1− rc), (C-15)
Appendix C - Expected Total Cost of PTSD Screening Process
175
pp_nc− = P{PCL is taken but its result remains unconfirmed|Patient is negative}
= rsβsrpu(1− rc). (C-16)
Thus, by substituting Eq. (C-13) and (C-14) into Eq. (C-12), and then by using the binomial
theorem given in Eq. (C-9) and after some algebra, the expected number of PCL tests per n
patients can be calculated as
E�Ip, n� = n�p �∑ i �pp_nc+ �
i−1pcon+ ∑ �(a+i−1)!
a!(i−1)!� �1 − rs(1− αs)rp�
aj−ia=0
ji=1 + 𝑗pp_nc
+ �1− pcon+ �
j−1�
+(1− p) �∑ i �pp_nc− �
i−1pcon− ∑ �(a+i−1)!
a!(i−1)!� �1 − rsβsrp�
aj−ia=0
ji=1 + 𝑗pp_nc
− �1− pcon− �
j−1��.
(C-17)
The variance in number of PCL tests per patient can be found in a similar manner, as explained
above. After some algebra, the following result for Var�Ip, n� can be obtained as
Var�Ip, n� = n�E�Ip2� − E2�Ip��
= n�p∑ i2 ��pp_nc+ �
i−1pcon+ ∑ �(a+i−1)!
a!(i−1)!� �1 − rs(1 − αs)rp�
aj−ia=0
ji=1
+ � j!i!(j−i)!
� �pp_nc+ �
i�1 − rs(1 − αs)rp�
j−i�
+(1− p)∑ i2 ��pp_nc− �
i−1pcon− ∑ �(a+i−1)!
a!(i−1)!� �1 − rsβsrp�
aj−ia=0 + � j!
i!(j−i)!� �pp_nc
− �i�1 − rsβsrp�
j−i�j
i=1
−�p �∑ i �pp_nc+ �
i−1pcon+ ∑ �(a+i−1)!
a!(i−1)!� �1 − rs(1− αs)rp�
aj−ia=0
ji=1 + 𝑗pp_nc
+ �1− pcon+ �
j−1�
+(1 − p) �∑ i �pp_nc− �
i−1pcon− ∑ �(a+i−1)!
a!(i−1)!� �1 − rsβsrp�
aj−ia=0
ji=1 + 𝑗pp_nc
− �1− pcon− �
j−1��
2
�.
(C-18)
Appendix C - Expected Total Cost of PTSD Screening Process
176
While Eq. (C-10), (C-11), (C-17), and (C-18), are rather lengthy expressions, special-purpose
software greatly facilitates their evaluation. Also note that the computer simulation results
presented elsewhere in this dissertation in Table 2-3 independently arrive at similar results, in
this sense confirming above derivations.
Expected number of CAPS tests per n patients:
The expected number of CAPS tests per n patients, E[Ic,n], simply is a function of the probability
that any single patient is checked using the CAPS test. As every patient is checked once using
the CAPS test or not checked using this test, this constitutes a Bernoulli trial and the distribution
of the number of CAPS tests per n patients therefore is binomial with binomial expectation
E�Ic, n� = n × P{Patient is checked using CAPS test}
= n × P{Result is confirmed by CAPS after 0 ≤ i < j − 1 years not having a confirmed result }
= n �p �rs(1 − αs)rpurc + �1 − pcon+ �rs(1− αs)rpurc + ⋯+ �1 − pcon
+ �(j−1)
rs(1 − αs)rpurc�
+(1− p) �rsβsrpurc + �1 − pcon− �rsβsrpurc + ⋯+ �1 − pcon
− �(j−1)
rsβsrpurc��
= n �p �rs(1 − αs)rpurc∑ �1 − pcon+ �
ij−1i=0 �+ (1 − p) �rsβsrpurc ∑ �1 − pcon
− �ij−1
i=0 ��
= n �p�rs(1− αs)rpurc �1−�1−pcon
+ �j
1−�1−pcon+ �
�� + (1 − p)�rsβsrpurc �1−�1−pcon
− �j
1−�1−pcon− �
���
= n �p�rs(1− αs)rpurc �1−�1−pcon
+ �j
rs(1−αs)rp�1−u(1−rc)��� + (1− p)�rsβsrpurc �
1−�1−pcon− �j
rsβsrp�1−u(1−rc)����
= n �p�urc �1−�1−pcon
+ �j
1−u(1−rc) �� + (1 − p)�urc �1−�1−pcon
− �j
1−u(1−rc) ���
= n �� urc1−u(1−rc)� �p �1 − �1 − pcon
+ �j� + (1− p) �1 − �1 − pcon
− �j���, (C-19)
Appendix C - Expected Total Cost of PTSD Screening Process
177
and with binomial variance Var[Ic,n]= n{∙}(1− {∙}), where {∙} denotes the term in square
brackets in Eq. (C-19) and is equal to the probability that any patient is checked using the CAPS
test:
Var�Ic, n� = n�� urc1−u(1−rc)� �p �1 − �1 − pcon
+ �j�+ (1 − p) �1 − �1 − pcon
− �j���
× �1 − �� urc1−u(1−rc)� �p �1 − �1 − pcon
+ �j�+ (1 − p) �1 − �1 − pcon
− �j����.
(C-20)
Expected number of false-positives per n patients:
The expected number of false-positives per n patients, E[FP], follows directly from the
probability of detecting a truly negative patient, qj, found in Eq. (B-2) of Appendix B:
E[FP] = n × P{Patient is negative} × P{Positive determination|Patient is negative}
= n(1 − p)(1− P{Negative determination|Patient is negative})
= n(1 − p) �1 − qj�
= n(1 − p)�1 − ��1−rsβsrp�1−u(1−rc)��
j�(1−u)βp+urcβc�+1−βp−u�1−βp−rc�1−βc��
1−u(1−rc) ��
= n(1 − p)��1−�1−rsβsrp�1−u(1−rc)��
j��(1−u)βp+urcβc�
1−u(1−rc) �. (C-21)
Expected number of false-negatives per n patients:
Similarly, the expected number of false-negatives per n patients, E[FN], follows directly from
the probability of detecting a truly positive patient, pj, found in Eq. (A-4) of Appendix A:
Appendix C - Expected Total Cost of PTSD Screening Process
178
E[FN] = n × P{Patient is positive} × P{Negative determination|Patient is positive}
= np(1 − P{Positive determination|Patient is positive})
= np �1 − pj�
= np�1 − ��1−�1−rs(1−αs)rp�1−u(1−rc)��
j��1−αp−u�1−αp−rc(1−αc)��
1−u(1−rc) ��
= np��1−rs(1−αs)rp�1−u(1−rc)��
j�1−αp−u�1−αp−rc(1−αc)��+(1−u)αp+urcαc
1−u(1−rc) �. (C-22)
Expected overall cost per n patients:
Now summing the products of each of the above five terms with their corresponding costs as
shown in Eq. (C-1) yields the expected total cost per n patients:
ECj = ksE�Is, n� + kpE�Ip, n� + kcE�Ic, n� + kFPE[FP] + kFNjE[FN]
= ksn�p �∑ i �ps_nc+ �
i−1pcon+ ∑ �(a+i−1)!
a!(i−1)!� (1 − rs)aj−i
a=0ji=1 + 𝑗ps_nc
+ �1− pcon+ �
j−1�
+(1− p) �∑ i �ps_nc− �
i−1pcon− ∑ �(a+i−1)!
a!(i−1)!� (1 − rs)aj−i
a=0ji=1 + 𝑗ps_nc
− �1 − pcon− �
j−1��
+kpn�p �∑ i �pp_nc+ �
i−1pcon+ ∑ �(a+i−1)!
a!(i−1)!� �1 − rs(1 − αs)rp�
aj−ia=0
ji=1 + 𝑗pp_nc
+ �1− pcon+ �
j−1�
+(1− p) �∑ i �pp_nc− �
i−1pcon− ∑ �(a+i−1)!
a!(i−1)!� �1 − rsβsrp�
aj−ia=0
ji=1 + 𝑗pp_nc
− �1− pcon− �
j−1��
+kcn � urc1−u(1−rc)� �p �1 − �1 − pcon
+ �j�+ (1 − p) �1 − �1 − pcon
− �j��
+kFPn(1 − p)��1−�1−rsβsrp�1−u(1−rc)��
j��(1−u)βp+urcβc�
1−u(1−rc) �
+kFNjnp��1−rs(1−αs)rp�1−u(1−rc)��
j�1−αp−u�1−αp−rc(1−αc)��+�(1−u)αp+urcαc�
1−u(1−rc) �. (C-23)
Appendix D - Overall Accuracy and Cost of TBI Screening Process
179
Appendix D - Overall Accuracy and Cost of TBI Screening Process
Notation and Assumptions:
αi = probability that any screening test i will determine a truly positive patient as negative,
αc = probability that the confirmation test will determine a truly positive patient as negative,
βi = probability that any screening test i will determine a truly negative patient as positive,
βc = probability that the confirmation test will determine a truly negative patient as positive,
r1 = probability that a truly positive patient has at least one teammate diagnosed with TBI,
r2 = probability that a truly negative patient has at least one teammate diagnosed with TBI,
ks = average cost of a screening test,
kc = average cost of the confirmation test,
kFP = average cost of a false-positive final result,
kFN = average cost of a false-negative final result, and
j = number of screening tests.
In the proposed screening process (given in Figure 2-18), after a negative determination by any
screening test i < j, the patient is scheduled for the next screening test i+1. If any screening test i
≤ j determines the patient as positive, he/she is sent to the confirmation test. If the patient, on the
other hand, is determined as negative by all j screening tests, it is checked whether or not any of
his teammates has diagnosed with TBI recently. If so, the patient is sent to the confirmation test.
Otherwise, no further action is taken and the patient is determined as negative. Note that this
decision stage is incorporated into the mathematical model using the rescreening rates of r1 and
Appendix D - Overall Accuracy and Cost of TBI Screening Process
180
r2 (explained above), and that r1 is assumed to be greater than r2. Other model assumptions
include
(1) Test sensitivities and specificities are assumed to remain constant over time.
(2) All tests are assumed to be independent of each other.
(3) The actual number of screening tests per any given patient, the expected number of
screening tests per positive patient and per negative patient are denoted, respectively, as
Is, E[Is | Positive] and E[Is | Negative]. Similarly, the actual number of confirmation tests
per any given patient, the expected number of confirmation tests per positive patient and
per negative patient are denoted, respectively, as Ic, E[Ic | Positive] and E[Ic | Negative].
(4) Screening and false diagnosis costs are assumed to remain constant over time.
Overall System Sensitivity:
A correct positive determination using j ≥ 1 screening tests can occur in the following manners:
(S1-, S2
-, …, Sj-, R, C+): – All j screening tests incorrectly classify patient as negative,
– Patient is sent to confirmation test, and
– Confirmation test correctly revises diagnosis to positive.
((S1-, S2
-, …, Si-1-, Si
+, C+), i ≤ j): – Some screening test i, 1 ≤ i ≤ j, correctly identifies patient as positive after i–1 ≥ 0 incorrect negative determinations,
– Patient is sent to confirmation test for review (always), and
– Confirmation test confirms diagnosis as positive.
Appendix D - Overall Accuracy and Cost of TBI Screening Process
181
The probability of detecting a truly positive patient using j screening tests, pj, then is the sum of
the probabilities of these events:
pj = Probability of detecting a truly positive patient using j screening tests
= P�S1−, S2
−, …, Sj−, R, C+� + P��S1
−, S2−, …, Si-1
−, Si
+, C+�i ≤ j�
= α1α2 … αjr1(1− αc) + (1 − α1)(1− αc) + α1(1 − α2)(1− αc) + ⋯ … + α1α2 … αj-1�1 − αj�(1 − αc)
= α1α2 … αjr1(1− αc) + �1 − α1α2 … αj-1αj�(1− αc)
= �1 −∏ αiji=1 (1 − r1)� (1− αc). (D-1)
Note that if all screening tests are assumed to have the same sensitivity, (1–αs), the above
expression in Eq. (D-1) simplifies to closed form as
pj = �1 − αsj(1 − r1)�(1− αc). (D-2)
Overall System Specificity:
Following the same logic above, a correct negative determination using j ≥ 1 screening tests can
occur in the following manners:
(S1-, S2
-, …, Sj-, R�): – All j screening tests correctly classify patient as negative, and
– Patient is not sent to confirmation test.
(S1-, S2
-, …, Sj-, R, C-): – All j screening tests correctly classify patient as negative,
– Patient is sent to confirmation test, and
– Confirmation test confirms patient as negative.
Appendix D - Overall Accuracy and Cost of TBI Screening Process
182
((S1-, S2
-, …, Si-1-, Si
+, C-), i ≤ j): – Some screening test i, 1 ≤ i ≤ j, incorrectly identifies patient as positive after i–1 ≥ 0 correct negative determinations,
– Patient is sent to confirmation test for review (always), and
– Confirmation test correctly revises diagnosis to negative.
Considering these events one at a time, the probability of detecting a truly negative patient using
j screening tests is
qj = Probability of identifying a truly negative patient as negative using j screening tests
= P{S1−, S2
−, …, Sj−, R�} + P{S1
−, S2−, …, Sj
−, R, C−} + P��S1−, S2
−, …, Si-1−
, Si+, C−�i ≤ j�
= ∏ �1 − βi�ji=1 (1− r2) + ∏ �1 − βi�
ji=1 r2�1− βc� + �1 −∏ �1 − βi�
ji=1 ��1− βc�
= 1 − βc �1 −∏ �1 − βi�ji=1 (1 − r2)�. (D-3)
As above, if all j screening tests are assumed to have similar specificities, (1–βs), the above
expression in Eq. (D-3) simplifies to closed form as
qj = 1 − βc �1 − �1 − βi�j(1− r2)�. (D-4)
Expected Total Cost:
The expected cost per n patients using j screening tests, ECj, is equal to the following sum:
ECj = Expected cost of all screening tests per n patients + Expected cost of all confirmation tests per n patients + Expected cost of all false-positives per n patients + Expected cost of all false-negatives per n patients
Appendix D - Overall Accuracy and Cost of TBI Screening Process
183
= (ks × Expected number of all screening tests per n patients) + (kc × Expected number of all confirmation tests per n patients) + �kFP × Expected number of false-positives per n patients� + �kFN × Expected number of false-negatives per n patients�, (D-5)
where each of the above four expectations are given separately below.
The expected number of screening tests per n patients, E[Is,n], is the sum of the expected number
of tests per single positive patient, E[Is | Positive], and the expected number of tests per single
negative patient, E[Is | Negative], conditioned as
E�Is, n� = n(P{Patient is positive}E[Is|Positive] + P{Patient is negative}E[Is|Negative]),(D-6)
where the expected number of screening tests for a positive patient is
E[Is|Positive] = ∑ iP{Is = i}ji=1
= 1P{Is = 1} + 2P{Is = 2} + ⋯+ (j− 1)P{Is = j − 1} + jP{Is = j}
= 1(1 − α1) + 2α1(1 − α2) + ⋯+ (j− 1)α1α2 … αj-2�1 − αj-1� + j�α1α2 … αj-1�1 − αj� + α1α2 … αj�
= (1 − α1) + ∑ �i∏ αki−1k=1 (1 − αi)�
j−1i=2 + j∏ αk
j−1k=1 , (D-7)
and similarly the expected number of screening tests for a negative patient is
E[Is|Negative] = ∑ iP{Is = i}ji=1
= 1P{Is = 1} + 2P{Is = 2} + ⋯+ (j− 1)P{Is = j − 1} + jP{Is = j}
= +1β1 + 2�1 − β1�β2 + ⋯+ (j− 1)�1 − β1��1− β2�… �1 − βj-2� βj-1
+j ��1 − β1��1− β2�… �1 − βj-1� βj + �1 − β1��1 − β2�… �1 − βj��
Appendix D - Overall Accuracy and Cost of TBI Screening Process
184
= β1 + ∑ �i∏ �1 − βk�i−1k=1 βi�
j−1i=2 + j∏ �1 − βk�
j−1k=1 . (D-8)
The expected number of confirmation tests per n patients, E[Ic,n], given below, simply is a
function of the probability that any single patient is checked using the confirmation test. As
every patient is screened once using the confirmation test or not screened using this test, this
constitutes a Bernoulli trial and the distribution of the number of confirmation tests per n patients
therefore is binomial with binomial expectation.
E�Ic, n� = n × P{Patient is screened by confirmation test}
= n(P{Some screening test determines patient as positive} +P{All screening tests determine patient as negative}P{Patient is sent to confirmation test})
= n �p�1 −∏ αiji=1 + ∏ αi
ji=1 r1� + (1 − p)�1 −∏ �1 − βi�
ji=1 + ∏ �1 − βi�
ji=1 r2��
= n �1 − �p∏ αiji=1 (1 − r1) + (1− p)∏ �1 − βi�
ji=1 (1− r2)��. (D-9)
The expected number of false-positives per n patients, E[FP], follows directly from the
probability of detecting a truly negative patient, qj, found in Eq. (D-3):
E[FP] = n × P{Patient is negative} × P{Positive determination|Patient is negative}
= n(1− p) �1 − qj�
= n(1− p)βc �1 −∏ �1 − βi�ji=1 (1 − r2)�. (D-10)
Similarly, the expected number of false-negatives per n patients, E[FN], follows directly from
the probability of detecting a truly positive patient, pj, found in Eq. (D-1):
E[FN] = n × P{Patient is positive} × P{Negative determination|Patient is positive}
Appendix D - Overall Accuracy and Cost of TBI Screening Process
185
= np �1 − pj�
= np �αc + (1 − αc)∏ αiji=1 (1− r1)�. (D-11)
Now summing the products of each of the above terms with their corresponding costs as shown
in Eq. (D-5) yields the expected total cost per n patients:
ECj = ksE�Is, n� + kcE�Ic, n� + kFPE[FP] + kFNE[FN]
= ksn �p �(1 − α1) + ∑ �i∏ αki−1k=1 (1 − αi)�
j−1i=2 + j∏ αk
j−1k=1 �
+(1− p)�β1 + ∑ �i∏ �1 − βk�i−1k=1 βi�
j−1i=2 + j∏ �1 − βk�
j−1k=1 ��
+kcn �1 − �p∏ αiji=1 (1 − r1) + (1 − p)∏ �1 − βi�
ji=1 (1 − r2)��
+kFPn(1 − p)βc �1 −∏ �1 − βi�ji=1 (1 − r2)�
+kFNnp �αc + (1− αc)∏ αiji=1 (1 − r1)�. (D-12)
If all screening tests can be assumed to have identical sensitivities (1–αi) = (1–αs) and identical
specificities (1–βi) = (1–βs) for all i, then the four expectations above in Eq. (D-12) simplify to
closed form and the total expected cost expression becomes
ECj = ksn�p �1−αsj
1−αs�+ (1 − p)�1−�1−βs�
j
βs��
+kcn�1 − �pαsj(1 − r1) + (1 − p)�1− βs�
j(1− r2)��
+kFPn(1 − p)βc �1 − �1 − βs�j(1 − r2)�
+kFNnp�αc + (1 − αc)αsj(1− r1)�. (D-13)