Feature Extraction and Selection for Prediction of ICU Patient’s Readmission Using Artificial Neural

Networks Ricardo Bento Afonso

Abstract— Patients’ readmissions in Intensive Care Unit (ICU) are associated with increased mortality, morbidity and costs. Cur-rent models for predicting ICU readmission have moderate predictive value. We postulate that greater predictive value can be achieved with physiological variables that can be assessed in the 24 hours before discharge. A data mining approach combining Artificial Neural Networks (ANN) with Sequential Forward Selection (SFS) was applied to a large retrospectively collected ICU database (MIMIC II), representing data from four different ICUs at Beth Israel Deaconess Medical Center, Boston. The goal of this study is to predict ICUs readmissions between 24 and 72 hours after discharge from ICU to help the development of clinical management plans that could potentially mitigate the risk of readmission and its associated problems. ANN combined with SFS was able to predict readmissions with an area under the receiver-operating curve (AUC) of 0.64 ± 0.10, a sensitivity of 0.72 ± 0.22, a specificity of 0.55 ± 0.06 and a accuracy of 0.57 ± 0.05. Variables selected as having the highest predictive power include: mini-mum of Albumin, minimum of Urine Output, minimum of Calcium, minimum of Platelets, Shannon entropy of Urine Output, minimum of temperature, maximum of Magnesium, mean of Urine Output, 50% weighted mean of Urine Output and maximum of Platelets.

Index Terms— Artificial Neural Networks, Feature Selection, Intensive Care Unit, Patient Readmission.

—————————— ——————————

1 INTRODUCTION

Patients readmitted to an intensive care unit during the same hospitalization have an increased risk of death (4-10 times more than non-readmitted patients), length of stay (LOS) (2-4 times more than non-readmitted patients), and higher costs [1–6]. Previous studies have demonstrated overall readmission rates of 4–14% [4, 6], of which nearly a third can be attributed to premature discharge from the critical care setting [3, 5]. In-creasing pressures on managing care and resources in ICUs is one explanation for strategies seeking to rapidly free expensive ICU beds, reducing the costs associated with patient stay. Faced with this scenario, a clinician may elect to discharge a patient currently in the ICU who has already had the benefits of stabi-lization and intensive monitoring, to make room for more acute patients allocated in the emergency department, exposing the outwardly transferring patients to the risk of readmission in the short term [7]. However if the patient is readmitted, not only the risk of death is higher, the LOS also increases, increasing the costs with it.

Current models for predicting ICU readmission have mod-erate predictive value. We postulate that greater predictive value can be achieved exclusively with physiological variables that can be assessed in the 24 hours before discharge.

A data mining approach combining Artificial Neural Net-works (ANN) with Sequential Forward Selection (SFS) was ap-plied to a large retrospectively collected ICU database (MIMIC II), representing data from four different ICUs at Beth Israel

Deaconess Medical Center, Boston. The goal of this study is to predict ICUs readmissions between 24 and 72 hours after dis-charge from ICU to help the development of clinical manage-ment plans that could potentially mitigate the risk of readmis-sion and its associated problems.

This work aims to help that in the future, models used to predict the risk of ICU patients’ readmissions, will be used as a decision help tool to help the clinicians avoid patients’ readmis-sions, improving not only the patients outcome and care given but also reducing the costs with the treatment (increasing the overall efficiency of the system).

This work aims to extract features from a database of phys-iological variables of the last 24 hours of stay of patients in the ICU, and later select which physiological variables are most predictive of readmissions within 24-72 hours, to be used in a Neural Network model (to predict readmissions of patients in the ICU).

Previous studies have examined different variables that are assessed at discharge, and that are considered to be predictive of readmission. They include fever, hypoxia, respiratory fail-ure, elevated respiratory rate, elevated heart rate, increasing age, ICU length of stay, proximity of extubation to time of dis-charge, need for organ support on the day of discharge, patient co-morbidities, ‘after-hours’ discharges and discharge to a high-dependency unit [1–3, 5, 8–18]. However, prediction mod-els based on these risk factors have moderate discrimination

ability (AUC ∈ [0.66 0.75]), and only performed slightly better than models based only upon APACHE II score (AUC = 0.63) at ICU admission [19].

To the best of the author’s knowledge, the only useful pre-dictive model based exclusively on physiological variables at ICU discharge was developed using fuzzy modeling (AUC = 0.72) [8]. The variables selected in this study as having the high-est predictive power include mean heart rate, mean tempera-ture, mean platelets, mean non-invasive arterial blood pressure (mean), mean spO2, and mean lactic acid, during the last 24 h before discharge.

To the best of the author’s knowledge, [10] is the only pre-dictive model applied in healthcare to predict and prevent un-planned ICU readmissions (for respiratory-related complica-tions) within 72 hours of discharge. The implementation of this model shows a 56.3% decrease of ICU readmissions over a year (September 2011 to September 2012). After the study they con-cluded that the implementation of their model brought: re-duced number of ICU readmissions due to respiratory-related complications; decreased hospital mortality; shortened hospital length of stay; reduced healthcare costs; increased ICU bed availability; enhanced multidisciplinary communication and collaboration.

Also of note is a study comprising 704,963 patients repre-senting 219 hospitals and 402 ICUs from throughout the United States (AUC = 0.71) [9], this study is important as it shows it’s possible using the same model to moderately predict readmis-sions to a vast number ICU’s.

The rest of this work is structured as follows. In Section 0 an overview of the methodologies used in this work is pre-sented. It begins by explaining the basics of feature selection and an overview of ANN’s. Section describes the use of the da-tabase (MIMIC II). Section presents the description of the em-pirical study and its assessment. Section presents the results from the empirical study described. In the end a discussion of the results obtained is presented. Finaly Section 7 presents the conclusions and recommendations for future work.

2 METHODS

2.1 Introduction

The MIMIC II database consists in a very large amount of clinical patient information, information that isn’t specific to any problem or study. So to extract useful knowledge hidden in all the information available and to use that knowledge effi-ciently and effectively for decision support, Knowledge Discov-

ery in Databases (KDD) can and should be used [20].

The KDD appeared because of an urgent need for a new generation of computational theories and tools to assist humans in extracting useful information (knowledge) from the rapidly growing volumes of digital data.

At an abstract level, the KDD field is concerned with the development of methods and techniques for making sense of data. The basic problem addressed by the KDD process is one of mapping low-level data (which are typically too voluminous to understand and easily understand) into other forms that might be more useful (for example, a predictive model for esti-mating the value of future cases). At the core of the process is

the application of specific data-mining methods for pattern dis-covery and extraction.

The KDD process can be decomposed into the following steps, as illustrated in Figure 1:

● Understand/define the problem - The first step involves defining the problem and determining the goals of the problem, taking into account current solutions to the problem if any.

● Data Acquisition - A target dataset is selected or created. ● Data Preprocessing - This phase includes, among other

tasks, noise removal/reduction, handling missing values and attribute discretization. This is one of the most important steps in the knowledge discovery process, since the accuracy of the models obtained from these datasets is strongly dependent on the data preprocessing. The goal is to improve the overall qual-ity of any information that may be discovered. This step often includes feature transformation which comprise methods that create new features (mean, maximum, minimum, etc).

● Feature Selection/Data Mining - Most datasets will con-tain a certain amount of redundancy that will not aid knowledge discovery and may in fact mislead the process. Fea-ture selection is a field of Machine Learning and Pattern Recog-nition and consists in reducing the dimensionality of the data by eliminating those features which are noisy, redundant or ir-relevant for a classification problem.

● Modeling - The choice of the modeling technique to be used may depend on many factors, including the source of the data set and the values that it contains.

● Interpretation/Evaluation - Once knowledge has been discovered, it is evaluated accordingly with the goals deter-mined in step one, with respect to validity, usefulness, novelty and simplicity. This may require repeating some of the previ-ous steps.

2.2 Feature Selection

Feature selection (FS) is a field of Machine Learning and Pattern Recognition (MLPR) and consists in reducing the dimensional-ity of the data by eliminating those features which are noisy, redundant or irrelevant for a classification problem, selecting a subset of relevant features for building robust learning models. The objectives of feature selection are manifold, the most im-portant ones being [21]: ● Avoid overfitting and improve model performance, i.e. prediction performance in the case of supervised classification and better cluster detection in the case of clustering. ● Provide faster and more cost-effective models. ● Provide a deeper insight into the underlying processes that generated the data, bringing to light new features that had not been previously considered for a given problem.

Figure 1: Overview of the steps that compose the KDD process

2-Data

Acquisition 1-Problem

Definition

3-Data

Preprocessing

4-Feature

Selection 5-Modeling 6-Interpretation

Databases Target DataPreprocessed

Data

Reduced

Data Patterns Knowledge

2.2.1 Sequential Forward Selection

For this thesis the Sequential Forward Selection (SFS) method was used.

The SFS is a type of greedy tree search feature selection, its main advantages it is simplicity and the possibility of graphical representation and transparent interpretation of the results which, for clinicians, is particular attractive. The main disad-vantage is related to the greedy and thus susceptible approach of finding local optima [22].

The SFS builds a model for each of the features in consider-ation and evaluating each feature using a performance criterion upon the test set (AUC is used in this study). The feature that returns the best value of the performance criterion is the one selected. Then, other feature candidates are added to the previ-ous best model, one at a time, and evaluated. Again, the combi-nation of features that maximizes the performance criterion is selected. When this second stage finishes, the model has two features. This procedure is repeated until the value of the per-formance criterion stops increasing. In the end, all the relevant features for the considered process should be obtained (Algo-rithm 1) [22].

Algorithm 2: Sequential Forward Selection

Let X be the vector of candidate features and Nt it’s size

Let Y be the vector of selected features and Nf it’s size

Let J be the performance criterion (AUC)

J∗ ←0

repeat

for i = 1→(Nt −Nf ) do

x∗ ←[]

Build a model using Y combined with xi // ith feature of X

Compute the Ji

if Ji > J ∗ then

J ∗ ← Ji

x∗ ← xi

end if

end for

Remove x∗ from vector X

Add x∗ to vector Y

until J ∗ stops improving or X is empty

2.3 Artificial Neural networks

Artificial Neural Networks (ANN) commonly referred as “neural networks”, are adaptive systems designed to simulate how human neurons are connected, replicating the highly com-plex, nonlinear and parallel information-processing system that the brain is. It resembles the brain in two aspects: ● The knowledge is acquired by the network from its envi-ronment through a learning process. ● Interneuron connection strengths, known as synaptic weights are used to store the acquired knowledge.

2.3.1 Artificial Neuron

A neuron is an information-processing unit that is funda-mental to the operation of a neural network. The block diagram of Erro! A origem da referência não foi encontrada. shows the

model of a neuron, which forms the basis for designing (artifi-cial) neural networks. ● There are three basic elements of the neuronal model [23]: A set of synapses or connecting links, each of which is charac-terized by a weight of its own. Specifically, a signal xj at the in-put of synapse j connected to neuron k is multiplied by the syn-aptic weight wkj. It is important to make a note of the manner in which the subscripts of the synaptic weight wkj, are written. The first subscript refers to the neuron in question and the second subscript refers to the input end of the synapse to which the weight refers. ● An adder for summing the input signals, weighted by the respective synapses of the neuron. The operations described here constitute a linear combiner. ● An activation function for limiting the amplitude of the output of a neuron. Typically, the normalized amplitude range of the output of a neuron is written, as the intervals [0,1] or [-1,1].

The neuronal model of Erro! A origem da referência não

foi encontrada. also includes an externally applied bias, de-noted by bk. The bias bk, has the effect of increasing or lowering the net input of the activation function, depending on whether it is positive or negative, respectively. In mathematical terms, we may describe a neuron k by writing the following pair of equations:

𝑢𝑘 = ∑ 𝑤𝑘𝑗𝑥𝑗𝑚𝑗=1 (1)

and

𝑦𝑘 = 𝜑(𝑢𝑘 + 𝑏𝑘) (2)

2.3.2 Bayesian Regulation Backpropagation

The Bayesian regulation backpropagation is a network training function that updates the weight and bias values ac-cording to Levenberg-Marquardt optimization. It minimizes a combination of squared errors and weights, and then deter-mines the correct combination so as to produce a network that generalizes well.

The Bayesian regularization can train any network as long as its weight, net input, and transfer functions have derivative functions. This training function doesn’t use validation data

wk1

wk2

wkj

.

.

.

Σ

Bias

bk

φ (vk)

Summing

junction

Synaptic

weights

Activation

function

Output

yk

x1

x2

xj

vk

.

.

.

Input

signals

Figure 2: Nonlinear model of a neuron

like some other training functions, it only use training and test-ing data.

Bayesian regularization minimizes a linear combination of squared errors and weights. It also modifies the linear combi-nation so that at the end of training the resulting network has good generalization qualities. For more detailed information on Bayesian regularization see [28, 29].

3 DATA

3.1 MIMIC II database

This study used data from the Multi-parameter Intelligent Monitoring for Intensive Care (MIMIC II) database [26]. This is a large database of ICU patients admitted to the Beth Israel Dea-coness Medical Center, collected from 2001 to 2006, and that has been de-identified by removal of all Protected Health Infor-mation. The MIMIC II database used (version 2.5) is formed by 26,655 patients, of which 19,075 are adults (>15 years old at time of admission). It includes high frequency sampled data of bed-side monitors, clinical data (laboratory tests, physicians’ and nurses’ notes, imaging reports, medications and other in-put/output events related to the patient) and demographic data. These adult patients experienced a total of 22,880 ICU-re-lated hospital admissions and a total of 25,852 ICU stays, giving an average of 1.36 ICU stays per patient. 38% of the adult pa-tients stayed at the medical ICU (MICU), 27% at the surgical ICU (SICU), 20% at the cardiac surgery recovery unit (CSRU) and 15% at the critical care unit (CCU) [27].

Inclusion criteria for our dataset included adult patients (>15 years) that were ICU inpatients for at least 24 h and read-missions back to any ICU of the same medical center between 24 and 72 h. This interval is often referred to as an early read-mission [28].

The reason for choosing 24 h as the lower bound for the readmission time window is related to how MIMIC II is struc-tured, patients readmitted to the ICU less than 24 h after their discharge are considered to belong to the same ICU stay, the patients might leave the ICU due to some treatment or medical examination.

The choice for 72 h as the upper bound for the readmission time window was based: ● In a study that found that readmission more than 72 h after initial discharge does not confer an increased in-hospital mor-tality [4] and so can be considered a different ICU stay ● In local clinical intensivist suggestions.

All included patients were also required to have at least one measurement of the variables shown in Table 1 during the last 24-h period within the ICU. This set of variables includes mon-itored variables laboratory tests and urine output.

Variables considered for modeling purposes are shown inTable 1. These variables were selected based on the hypothe-sis that a good predictive value can be achieved using a few physiological variables and, taking into account the following directives [8]: ● Variables must be easily and/or routinely assessed in the 24 h before discharge [29]. ● A balance has to exist in the number of selected variables given that it will affect the number of patients that will form the dataset, i.e. the more variables that are defined, the fewer the

patients that are likely to have all of them collected at the same time. ● Selecting a high number of variables may bias the dataset towards selecting patients having similar conditions that re-quired their specific measurement/testing; ● The variables chosen should be independent with minimal correlation.

The patient selection process is summarized in Figure 3, with the selected patients’ characteristics in Table 2. Table 1: List of variables considered from MIMIC II

Type of variable

Variable name (units) Average sampling frequency (samples

per day)

Monito-ring

signals

Heart rate (beats/min) 27.25 Respiratory rate (breaths/ min) 26.54 Temperature (ºC) 9.63 SpO2 (%) 26.71

Non-invasive arterial Blood pressure (systolic) (mmHg)

25.72

Non-invasive arterial Blood pressure (mean) (mmHg)

25.56

Labora-tory tests

Red blood cell count (cells × 10³/µL)

3.15

White blood cell count (cells × 10³/µL)

3.04

Platelets (cells × 10³/µL) 2.83 Hematocrit (%) 2.78 BUN (mg/dL) 2.16 Sodium (mg/dL) 4.37 Potassium (mg/dL) 3.28 Calcium (mg/dL) 2.56 Chloride (mg/dL) 1.68 Creatinine (mg/dL) 2.94 Magnesium (mg/dL) 1.66 Albumin (g/dL) 1.86 Arterial pH 1.72 Arterial base excess (mEq/L) 1.68 Lactic acid (mg/dL) 2.28

Other Urine output (mL/h) 22.25

Table 2: Patient characteristics for the selected dataset

3.2 Feature Extraction

The time series representing each of the variables present in Table 1 were used to calculate their respective arithmetic mean, standard deviation, maximum, minimum, weighted mean (five different linear weighted means with weights shown in Figure 4) and Shannon and Log Energy entropies values during the last 24 h of an ICU stay of a patient.

The five weighted means are calculated (3) so that the phys-

iological variables (xi) 24 hours before the patient discharge have a weight (wi) of 10%, 30%, 50%, 70% and 90% when com-pared with the weight at the patient discharge.

�̅� =∑ 𝑤𝑖𝑥𝑖

𝑛𝑖=1

∑ 𝑤𝑖𝑛𝑖=1

(3)

The weighted mean was added to the physiological varia-

bles transformations studied in [8], to assess if there is improve-

ment in the predictability of ICU readmissions over the use of

the arithmetic mean, since the last doesn’t account for the evo-

lution of the physiological variable (if increases or decreases in

time).

For the entropy two different criteria were used: In the following expressions, (𝑠) is the signal and (𝑠𝑖) the

coefficients of (𝑠) in an orthonormal basis. The entropy E must be an additive cost function such that

𝐸(0) = 0 and

𝐸(𝑠) = ∑ 𝐸(𝑠𝑖)𝑖 (4)

The (non-normalized) Shannon entropy:

𝐸1(𝑠𝑖) = 𝑠𝑖2 log(𝑠𝑖

2) (5)

so

𝐸1(𝑠) = − ∑ 𝑠𝑖2 log(𝑠𝑖

2)𝑖 (6)

with the convention 0𝑙𝑜𝑔(0) = 0.

The "log energy" entropy:

𝐸2(𝑠𝑖) = log(𝑠𝑖2) (7)

All patients Patients

readmitted to the ICU

Patients not readmitted to the ICU

Number of patients, n

1028 135 893

Age, yr 61.5 ± 18.0 58.7 ± 18.1 60.6 ± 18.7

Males, n 585 68 517

Females, n 443 67 376 Hospital LOS,

days 23.5 ± 19.5 25.4 ± 19.8 25.1 ± 20.3

SAPS I 17.3 ± 4.9 17.5 ± 5.2 16.9 ± 4.6 Weekend dis-

charge, n 262 35 227

Night discharge, n 53 7 46

Measurements taken per patient in last 24h before

13.3 ± 9.0 12.7 ± 9.1 13.4 ± 9.0

Patients with one measurement in

last 24h 258 37 221

Figure 3: Patient selection scheme

0

0,2

0,4

0,6

0,8

1

24 12 0

We

igh

ts

Hours to the ICU Discharge

10% 30% 50% 70% 90%

Figure 4: Variation of weights for the weighted mean

so

𝐸2(𝑠) = ∑ log(𝑠𝑖2)𝑖 (8)

so with the convention 𝑙𝑜𝑔(0) = 0. With this the entropy can compress any signal (time series) or image and use the end result for modeling purposes. There are a great number of studies that use entropy as feature extrac-tion [30–33].

To the best of the author’s knowledge, no study used either the weighted mean or the entropy on the physiological varia-bles transformations for feature extraction at predicting ICU re-admissions.

3.3 Data Preprocessing

As with other real-world databases, a few preprocessing steps were necessary to improve the quality of the data. In order to deal with variables collected with different sampling periods, a template variable was used. This process aligns all samples to the same point in time as a designated template variable. Heart rate was chosen as the template variable on the basis that this is the most frequently measured variables (as can be seen in Table 1)and thus, introduces fewer artifacts in the data.

With regards to missing data, in general, ICU data can be missing either because they are perceived to be irrelevant for the current clinical problems (thus, not recorded) or because ex-ogenous interventions or endogenous activities have rendered the data useless, due to disconnections, sensor errors, equip-ment changes, intrusive diagnostics and request-based data (such as blood tests) [34]. Data missing for an intentional reason (e.g. patient is transported out the ICU for an imaging scan) was considered non-recoverable and thus deleted.

On the other hand, data missing for some unintentional

reason (e.g. sensor goes off patient’s chest) was considered re-coverable and the last available value was used to impute val-ues to these segments [35–37]. Outliers were also removed from the data.

4 MODEL SETUP

4.1 Model Construction

The dataset was randomly divided into two equal parts,

keeping the ratio between classes: one for feature selection (FS subset) and the other for model assessment (MA subset). This was done, in a first phase, to select the best model features and in a second phase, to assess its performance over an independ-ent dataset not used during any step of FS.

The FS used a deterministic wrapper method (Sequential For-

ward Selection), in which the classifier used was a feedforward backpropagation Bayesian regulation ANN, with a varying number of neurons in one hidden layer and with one neuron on the output layer.

Given that this is a classification problem, and that we have a linear consequent, a threshold t is required to turn the contin-uous output y ∈ [0,1] into the binary output y ∈ {0,1}. In this way, if y < t then y = 0, and if y ≥ t then y =1.

In the FS subset, SFS was performed by randomly partition the data into training (60%) and testing (40%) fractions, also keeping the ratio between classes in each data partition. The features tested where the mean, minimum, maximum, standard deviation, one of the weighted means , the Log Energy Entropy and the Shannon Entropy for every physiological variable in Table 1. For this reason five different sets of features were tested, one for each different weighted mean.

The subset of features resulting in the highest value of area under receiver-operating curve (AUC) for the FS test fraction was selected as containing the most predictive variables to be used in the MA subset.

The threshold used for the determination of each patient class, was found by iteratively maximizing the AUC in the training part of the FS subset.

The MA subset was then defined in such a way to contain only these most predictive variables selected in the previous stage and its threshold.

A 10-fold cross-validation was performed to assess the best model. The 10-fold cross validation method consists in dividing the data set in 10 equally sized partitions, and per-forming 10 classification experiments for calculating their performance. For each classification, nine of the partitions are put together and used for training (building the classifier), and the other one partition is used for testing. In the next clas-sification, another partition is designed for testing the classi-fier built with the rest of the partitions. Ten classification ex-periments are performed so that each partition is used for testing a classifier. The data partitioning is randomly per-formed once, keeping the ratio between classes in each parti-tion. The AUC, accuracy, sensitivity and specificity results in the 10-fold cross validation stage were averaged over the rounds. This whole process was repeated ten times (Erro! A origem da referência não foi encontrada.), due to the randomness

process of the ANN.

All Data

SFS:

60% training

40% testing

10-fold cross-

validation:

90% training

10% testing

50% 50%

Best Model

Performance (Best

AUC in the testing

data)

Model assessement

Best features set

and threshold

× 10

Figure 5: Process scheme

4.2 Model Assessment

Since this work consisted of assigning a combination of se-lected features to one of two possible classes (patient will be re-admitted or patient will not be readmitted), the AUC could be used to assess its discrimination performance [38]. This is a function of the true positive ratio versus the false positive ratio, integrated over all thresholds. The true positive rate and true negative rate correspond to the sensitivity (Eq.9) and specificity (Eq. 10) of the model and in this particular problem, represent the cases where the patient was correctly classified as being re-admitted and the cases where the patient was correctly classi-fied as not being readmitted: ● True Positive (TP) = correctly readmitted ● False Positive (FP) = incorrectly readmitted ● True Negative (TN) = correctly non-readmitted ● False Negative (FN) = incorrectly non-readmitted

𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 =𝑇𝑃

𝑇𝑃+𝐹𝑁 (9)

𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 =𝑇𝑁

𝑇𝑁+𝐹𝑃 (10)

The accuracy of the model is given by the equation 11 and

represents the proportion of true results in the population.

𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 =𝑇𝑃+𝑇𝑁

𝑇𝑃+𝐹𝑃+𝑇𝑁+𝐹𝑁 (11)

5 RESULTS

ANN combined with SFS was able to predict readmissions with an area under the receiver-operating curve (AUC) of 0.64 ± 0.10, a sensitivity of 0.72 ± 0.22, a specificity of 0.55 ± 0.06 and a ac-curacy of 0.57 ± 0.05. The Most predictive variables selected are in Table 3. The optimum value for the threshold, to turn the continuous output y into the binary output y ∈{0,1} was found to be 0.13, i.e. if y < 0. 13 then y = 0, and if y ≥ 0. 13 then y = 1. Table 3: Most predictive features selected from the best AUC in the 10-fold cross-validation

Most predictable variables se-lected Minimum of Albumin

Minimum of Urine Output

Minimum of Calcium

Minimum of Platelets

Shannon entropy of Urine Out-put Minimum of temperature

Maximum of Magnesium

Mean of Urine Output

50% weighted mean of Urine Output Maximum of Platelets

6 DISCUSSION

As can be seen in Table 3, despite the fact that the best solu-tion found as 10 features, only 6 physiological variables are used, two of them repeating themselves with different transfor-mations (Platelets repeats two times and Urine Output repeats four times).

In Figure 6 can be seen that platelets and urine output are the two most selected variables between the best studies, which is no surprise the difference between the mean of the two clas-ses for this two variables (Platelets: 214.81 for readmissions vs. 248.36 for non-readmissions; Urine Output: 94.85 for readmis-sions vs. 107.40 for non-readmissions).

The most selected feature transformations between the best studies were (Figure 7): minimum, maximum and Shannon en-tropy. The Log Energy entropy wasn’t selected once in the best tests.

This work found a new combinations of physiological vari-ables not previously associated with the prediction of ICU re-admissions [1–3, 5, 8–18], despite some variable crossover with this articles.

The observed sensitivity was significantly higher than spec-ificity, meaning that the ratio of patients incorrectly classified as non-readmitted (false negatives) was significantly lower than the ratio of patients incorrectly classified as readmitted

0

10

20

30

40

50

60

Nu

mb

er

of

Ocu

rre

nci

es

Figure 6: Physiological variables frequency from studies

01020304050607080

Nu

mb

er

of

Ocu

rre

nci

es

Figure 7: Features transformation frequency from studies

(false positives). This high rate of false positives might be ex-plained by the fact that in some cases, a patient was not read-mitted to the ICU only because of the limit availability of ICU beds, in detriment of more critical patients. This fact clearly bias the data and might be the cause of the moderately predictability of the models so far, when compared with clinical studies where the output is not subjected to the clinicians subjectivity and limited resources (as an example, studies that predict ICU patients’ death [9]).

Other problems that might occur and are not contemplated in the data are: irresolvable clinical scenarios associated with not-for-resuscitation policies that could lead to comfort cares in non-ICU environments; transfer to other hospitals, and other intangible features not directly ascertainable from data driven exploratory processes. Scenarios such as these lead the models to be incorrectly trained with subsequent misleading classifica-tions.

There are some limitations in the approach we have used: models do not take into account the nature and quality of care that readmitted patients receive outside of the ICU and that clinical progress notes were not assessed for potential explana-tion of both discharge and readmissions.

7 CONCLUSION

The best average AUC found was of 0.64 and the corre-spondent accuracy of 0.57.

Recognizing and managing patients at high risk of ICU re-admission is important for maximizing patient outcomes and minimizing costs. In this study, a prediction algorithm based on ANN modeling combined with sequential forward selection is proposed. Using this methodology we show that it is possible to use a small number of physiological variables (Albumin, Urine Output, Calcium, Platelets, Temperature and Magne-sium) in the 24 hours before discharge to predict readmitted pa-tients (average sensitivity = 0.72). However some of this physi-ological variables need more than one feature transformation to predict the model (Platelets repeats two times and Urine Out-put repeats four times).

The model didn’t perform so well at predicting non-read-mitted patients (average specificity = 0.55), this might be be-cause of some data bias (clinicians subjectivity and limited re-sources) and other scenarios not directly ascertainable from the data. For future work a multi-model or a multi-stage model approach might improve performance. Due to the severe state of the ICU patients, it’s important that future work should be more fo-cused in non-invasive tests/signals (monitoring signals and others). Not only improves patients’ condition by reducing the invasive tests needed, but also gives us more data points (sam-pled at higher frequency) closely before discharge, abeling With this, a NARX model could be developed to run online with the data acquisition monitors to give a real time risk of re-admission. Creating a score for the risk of ICU readmissions similar to other existing medical scores, to help clinicians in their assessment should also be considered.

ACKNOWLEDGMENTS

My first words of appreciation are undoubtedly to my supervi-sors Professora Susana Vieira and Professor João Sousa. I would like to thank them for their invaluable support and extreme pa-tience, and for the advice and orientation provided throughout this work.

My colleagues also deserve a special acknowledgment due to all the advice, support and information they provided, through the course and especially in these last months.

REFERENCES

[1] J. Renton, D. V Pilcher, J. D. Santamaria, P. Stow, M. Bailey, G. Hart, and G. Duke, “Factors associated with increased risk of readmission to intensive care in Australia.,” Intensive care medicine, vol. 37, no. 11, pp. 1800–1808, Nov. 2011.

[2] E. Boudesteijn, S. Arbous, and P. van den Berg, “Predictors of intensive care unit readmission within 48 hours after discharge,” Critical Care, vol. 11, no. Suppl 2, p. P475, 2007.

[3] C. G. Durbin and R. F. Kopel, “A case-control study of patients readmitted to the intensive care unit.,” Critical care medicine, vol. 21, no. 10, pp. 1547–1553, Oct. 1993.

[4] A. L. Rosenberg and C. Watts, “Patients readmitted to ICUs* : a systematic review of risk factors and outcomes.,” Chest, vol. 118, no. 2, pp. 492–502, Aug. 2000.

[5] W. Baigelman, R. Katz, and G. Geary, “Patient readmission to critical care units during the same hospitalization at a community teaching hospital.,” Intensive care medicine, vol. 9, no. 5, pp. 253–256, Jan. 1983.

[6] M. Elliott, “Readmission to intensive care: a review of the literature.,” Australian critical care : official journal of the Confederation of Australian Critical Care Nurses, vol. 19, no. 3, pp. 96–98, 100–104, Aug. 2006.

[7] D. B. Chalfin, S. Trzeciak, A. Likourezos, B. M. Baumann, and R. P. Dellinger, “Impact of delayed transfer of critically ill patients from the emergency department to the intensive care unit.,” Critical care medicine, vol. 35, no. 6, pp. 1477–1483, Jun. 2007.

[8] A. S. Fialho, F. Cismondi, S. M. Vieira, S. R. Reti, J. M. C. Sousa, and S. N. Finkelstein, “Data mining using clinical physiology at discharge to predict ICU readmissions,” Expert Systems with Applications, vol. 39, no. 18, pp. 13158–13165, Dec. 2012.

[9] O. Badawi and M. J. Breslow, “Readmissions and death after ICU discharge: development and validation of two predictive models.,” PloS one, vol. 7, no. 11, p. e48758, Jan. 2012.

[10] C. Grottenthaler and C. Reese, “Reducing Icu Readmission,” Critical Care Medicine, vol. 40, no. 12, 2012.

[11] L. M. Chen, C. M. Martin, S. P. Keenan, and W. J. Sibbald, “Patients readmitted to the intensive care unit during the same hospitalization: clinical features and outcomes.,” Critical care medicine, vol. 26, no. 11, pp. 1834–1841, Nov. 1998.

[12] G. S. Cooper, C. A. Sirio, A. J. Rotondi, L. B. Shepardson, and G. E. Rosenthal, “Are readmissions to the intensive care unit a

useful measure of hospital performance?,” Medical care, vol. 37, no. 4, pp. 399–408, Apr. 1999.

[13] C. Franklin and D. Jackson, “Discharge decision-making in a medical ICU: characteristics of unexpected readmissions.,” Critical care medicine, vol. 11, no. 2, pp. 61–66, Feb. 1983.

[14] C. Goldfrad and K. Rowan, “Consequences of discharges from intensive care at night.,” Lancet, vol. 355, no. 9210, pp. 1138–1142, Apr. 2000.

[15] P. G. H. Metnitz, F. Fieux, B. Jordan, T. Lang, R. Moreno, and J.-R. Le Gall, “Critically ill patients readmitted to intensive care units--lessons to learn?,” Intensive care medicine, vol. 29, no. 2, pp. 241–248, Feb. 2003.

[16] A. L. Rosenberg, T. P. Hofer, R. A. Hayward, C. Strachan, and C. M. Watts, “Who bounces back? Physiologic and other predictors of intensive care unit readmission.,” Critical care medicine, vol. 29, no. 3, pp. 511–518, Mar. 2001.

[17] H. B. Rubins and M. A. Moskowitz, “Discharge decision-making in a medical intensive care unit. Identifying patients at high risk of unexpected death or unit readmission.,” The American journal of medicine, vol. 84, no. 5, pp. 863–869, May 1988.

[18] N. Snow, K. T. Bergin, and T. P. Horrigan, “Readmission of patients to the surgical intensive care unit: patient profiles and possibilities for prevention.,” Critical care medicine, vol. 13, no. 11, pp. 961–964, Nov. 1985.

[19] A. J. Campbell, J. A. Cook, G. Adey, and B. H. Cuthbertson, “Predicting death and readmission after intensive care discharge.,” British journal of anaesthesia, vol. 100, no. 5, pp. 656–662, May 2008.

[20] U. Fayyad, G. Piatetsky-shapiro, and P. Smyth, “From Data Mining to Knowledge Discovery in,” pp. 37–54, 1996.

[21] Y. Saeys, I. Inza, and P. Larrañaga, “A review of feature selection techniques in bioinformatics.,” Bioinformatics (Oxford, England), vol. 23, no. 19, pp. 2507–2517, Oct. 2007.

[22] L. F. Mendonça, S. M. Vieira, and J. M. C. Sousa, “Decision tree search methods in fuzzy modeling and classification,” International Journal of Approximate Reasoning, vol. 44, no. 2, pp. 106–123, Feb. 2007.

[23] S. Haykin, Neural Networks - A Comprehensive Foundation, 9th ed. Pearson Education.

[24] D. J. C. MacKay, “Bayesian Interpolation,” Neural Computation, vol. 4, no. 3, pp. 415–447, May 1992.

[25] Foresee and Hagan, “No Title,” in Proceedings of the InternationalJoint Conference on Neural Networks, 1997.

[26] M. Saeed, M. Villarroel, A. T. Reisner, G. Clifford, L.-W. Lehman, G. Moody, T. Heldt, T. H. Kyaw, B. Moody, and R. G. Mark, “Multiparameter Intelligent Monitoring in Intensive Care II: a public-access intensive care unit database.,” Critical care medicine, vol. 39, no. 5, pp. 952–960, May 2011.

[27] M. Saeed, C. Lieu, G. Raber, and R. G. Mark, “MIMIC II: a massive temporal ICU patient database to support research in

intelligent patient monitoring.,” Computers in cardiology, vol. 29, pp. 641–644, Jan. 2002.

[28] D. C. Angus, “Grappling with intensive care unit quality--does the readmission rate tell us anything?,” Critical care medicine, vol. 26, no. 11, pp. 1779–1780, Nov. 1998.

[29] S. of C. C. Medicine, “Guidelines for ICU Admission , Discharge , and Triage,” Critical Care Medicine, vol. 27, no. March, pp. 633–638, 1999.

[30] A. Partap and B. Singh, “Shannon and Non-Shannon Measures of Entropy for Statistical Texture Feature Extraction in Digitized Mammograms,” WCECS, vol. II, 2009.

[31] J. Dayou, N. C. Han, H. C. Mun, and A. H. Ahmad, “Classification and Identification of Frog Sound Based on Entropy Approach,” IPCBEE, vol. 3, pp. 184–187, 2011.

[32] M.-Y. Lee and C.-S. Yang, “Entropy-based feature extraction and decision tree induction for breast cancer diagnosis with standardized thermograph images.,” Computer methods and programs in biomedicine, vol. 100, no. 3, pp. 269–282, Dec. 2010.

[33] C. Fu and F. Dong, “Wavelet packet entropy feature extraction and characteristics analysis for gas/liquid two-phase flow regimes,” in 2011 IEEE International Instrumentation and Measurement Technology Conference, 2011, pp. 1–5.

[34] G. D. Clifford, W. J. Long, G. B. Moody, and P. Szolovits, “Robust parameter extraction for decision support using multimodal intensive care data.,” Philosophical transactions. Series A, Mathematical, physical, and engineering sciences, vol. 367, no. 1887, pp. 411–429, Jan. 2009.

[35] P. D. Allison, Missing Data (Quantitative Applications in the Social Sciences). SAGE Publications, Inc, 2001, p. 104.

[36] F. Cismondi, A. S. Fialho, S. M. Vieira, J. M. C. Sousa, S. R. Reti, M. D. Howell, and S. N. Finkelstein, “Computational intelligence methods for processing misaligned, unevenly sampled time series containing missing data,” in 2011 IEEE Symposium on Computational Intelligence and Data Mining (CIDM), 2011, pp. 224–231.

[37] A. Fialho, F. Cismondi, S. Vieira, J. Sousa, S. Reti, R. Welsch, M. Howell, and S. Finkelstein, “Missing Data In Large Icu Databases,” Critical Care Medicine, vol. 38, no. 12, p. 16, 2010.

[38] E. W. Steyerberg, A. J. Vickers, N. R. Cook, T. Gerds, M. Gonen, N. Obuchowski, M. J. Pencina, and M. W. Kattan, “Assessing the performance of prediction models: a framework for traditional and novel measures.,” Epidemiology (Cambridge, Mass.), vol. 21, no. 1, pp. 128–138, Jan. 2010.