microsimulation of organ dysfunction › ~cler › mscmp3780 › microsimulation_fina… · web...

Dynamic microsimulation to model multiple outcomes in cohorts of critically ill patients

Gilles Clermont*, MD, MScVladimir Kaplan*, MD

Rui Moreno†, MDJean-Louis Vincent‡, MD, PhD

Walter T. Linde-Zwirble§

Ben Van Hout¢, PhDDerek C. Angus*¶, MD, MPH

* Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, PA† Intensive Care Medicine, Hospital de St. Antonio dos Capuchos, Lisbon, Portugal‡ Department of Intensive Care, Erasmus University Hospital, Brussels, Belgium§ Health Process Management, Inc, Doylestown, PA¢ Department of Health Care Policy and Management, Erasmus University,

Rotterdam, Netherlands¶ Center for Research on Health Care and Graduate School of Public Health,

University of Pittsburgh, Pittsburgh, PA

Running headMicrosimulation of organ dysfunction

Word count3,239

Financial supportPartially supported by Eli Lilly & Company (Gilles Clermont and Derek C. Angus) and by the Stiefel-Zangger Foundation, University of Zurich, Switzerland (Vladimir Kaplan)

Address for correspondenceGilles Clermont, MD, MScRoom 606B, Scaife HallCritical Care MedicineUniversity of Pittsburgh3550 Terrace StreetPittsburgh, PA 15261

Tel: (412) 647 7980Fax: (412) 647 3791E-mail: [email protected]

Microsimulation of organ dysfunction

Abstract

Background: Existing intensive care unit (ICU) prediction tools forecast

single outcomes, (e.g., risk of death) and do not provide information on timing.

Objective: To build a model that predicts the temporal patterns of multiple

outcomes, such as survival, organ dysfunction, and ICU length of stay, from the

profile of organ dysfunction observed on admission.

Design: Dynamic microsimulation of a cohort of ICU patients.

Setting: 49 ICUs in 11 countries.

Patients: 1,449 patients admitted to the ICU in May 1995.

Interventions: None.

Model Construction: We developed the model on all patients (n=989)

from 37 randomly-selected ICUs using daily Sequential Organ Function Assessment

(SOFA) scores. We validated the model on all patients (n=460) from the remaining

12 ICUs, comparing predicted-to-actual ICU mortality, SOFA scores, and ICU length

of stay (LOS).

Main Results: In the validation cohort, the predicted and actual mortality

were 20.1% (95%CI: 16.2%-24.0%) and 19.9% at 30 days. The predicted and

actual mean ICU LOS were 7.7 (7.0-8.3) and 8.1 (7.4-8.8) days, leading to a 5.5%

underestimation of total ICU bed-days. The predicted and actual cumulative SOFA

scores per patient were 45.2 (39.8-50.6) and 48.2 (41.6-54.8). Predicted and actual

mean daily SOFA scores were close (5.1 vs 5.5, P=0.32). Several organ-organ

interactions were significant. Cardiovascular dysfunction was most, and

neurological dysfunction was least, linked to scores in other organ systems.

Conclusions: Dynamic microsimulation can predict the time course of

multiple short-term outcomes in cohorts of critical illness from the profile of organ

dysfunction observed on admission. Such a technique may prove practical as a

2


prediction tool that evaluates ICU performance on additional dimensions besides

the risk of death.

3


Descriptor

Severity-of-disease scoring systems

Keywords

Intensive care, multiple organ failure, mortality, resource use, computer

simulation, microsimulation

4


Progress in acute care medicine and new resuscitation techniques have led

to significant improvement in the immediate survival of victims of severe trauma,

burns, and infections. However, early resuscitation is often followed by progressive

dysfunction of multiple organ systems (MODS) [1, 2] that may lead to prolonged

morbidity and death [3, 4]. Indeed, MODS accounts for most late-onset deaths in

critical illness [5] and consumes large amounts of healthcare resources [6].

Several investigators have developed schemes to quantify organ dysfunction

and have consistently demonstrated a close relationship between the presence and

intensity of MODS and hospital mortality [7-9]. Specifically, the cumulative burden

of organ failure in terms of both, the number of organs failing [8, 10] and the

degree of organ dysfunction within each organ system [11] was the strongest

predictor of death [12]. However, these prediction tools typically predict single

outcomes (e.g., risk of death) at fixed time points. Other statistical techniques

provide prediction the timing of events [13, 14], but only simulations provide

simultaneous predictions for the incidence and timing of multiple outcomes.

Our first objective was to build a single model that predicts, in a cohort of

critically ill patients, the temporal patterns of multiple outcomes, such as survival,

organ dysfunction, and intensive care unit (ICU) length of stay, from demographic

variables and the profile of organ dysfunction assessed on admission by the

sequential organ dysfunction score (SOFA) [15]. Our second objective was to use

the model to explore organ-organ interactions. Because such predictions cannot

typically be constructed using standard analytic methods[16], we developed a

microsimulation model, a technique suited to predict multiple events over time in

systems where characteristics change in a time dependent fashion. We validated its

predictive performance in a separate group of patients using basic demographic

5


data and the first ICU day SOFA score only.

Such simulations have a range of potential applications in the ICU such as

predicting the rate and timing of various events and outcomes, and the potential

impact of interventions aimed at modifying the predictors of these outcomes.

6


Materials and methods

Patient population

We used an international database of 1449 critically ill patients from 40

institutions (49 ICUs) in 11 countries. The database included all adult patients

admitted to the ICUs in May 1995, except for those who stayed in the ICU for less

than 48 hours after uncomplicated surgery. The data were prospectively collected

by the European Society of Intensive Care Medicine (ESCIM) to evaluate and

validate the usefulness of the SOFA score [8, 17]. SOFA scores were collected daily

until ICU discharge or a maximum of 33 days. Details regarding data collection

were described previously [8].

Missing values

Scores missing on days prior to the first recorded value were attributed the

first available score. Scores missing between the last recorded score and ICU

discharge were attributed the last recorded score. Other missing scores were

assigned according to the following rules: linear interpolation was used for organ

systems with a slowly evolving physiology (hematologic, renal, and hepatic) and

last available scores were carried forward for the other organ systems

(cardiovascular, pulmonary, and neurologic). We chose not to implement a priori

stochastic rules[18] of imputing missing data a because of the expected non-

randomness and high predictability of missing values [19]. To assess the sensitivity

of predictions to different imputation rules for missing data, we provide predictions

using last value carried forward and next value carried backwards as alternatives

imputation rules for missing intercurrent values.

7


Dynamic microsimulation

Dynamic microsimulation is a method particularly suitable for modeling

multiple events over time that occur in systems where the interactions within the

system are complex, the characteristics of the systems change in a time-dependent

fashion, and the analysis of the system is intractable by conventional analytic

methods [16]. Such models allow probabilistic projections forward in time on

cohorts with known baseline characteristics. In our model, the patient represents

the complex system, defined by his/her profile of organ dysfunction, which evolves

over time and dictates the occurrence of multiple outcomes (i.e., death, organ

failure, or ICU discharge). Microsimulation is well-suited to describe cohort

behavior, and not the time course of individual patients.

Model development

We used a subset of 989 patients from 37 randomly selected ICUs to develop

the microsimulation model. We built the model in three steps. First, using the

SOFA scores of the current day, we constructed trinomial logistic regression

equations to generate daily probabilities of “discharge from the ICU on the current

day”, “ICU death on the current day”, or “remain in the ICU until the next day” (the

ternary outcome sub-model). Because we anticipated that the predictive

probabilities of a given pattern of SOFA scores would change over time, we also

included an explicit time factor as independent predictor (periods A, B and C

corresponding to ICU day 1, ICU days 2 to 9, and ICU days 10 to 30). We ignored

data beyond 30 days because of a paucity of data points. We also included sex,

type of patient (emergent/scheduled surgery, trauma, medical/cardiac/others), age

(<45, 45-64, >65) as predictors. Second, to update SOFA scores in individual

patients remaining in the ICU, we constructed multinomial logistic regression

8


equations to generate SOFA scores for the next day based on SOFA scores of the

current day as well as the same demographic predictors described above and ICU

day (the SOFA sub-models). We developed 6 SOFA sub-models (one for each organ

systems). Third, we integrated all sub-models in a global model, a dynamic

microsimulation, and propelled each patient in daily steps until ICU discharge or a

maximum of 30 days.

To verify the ability of the microsimulation model to reproduce the outcome

and the level of organ dysfunction, we simulated the time course of the ICU stay in

the development cohorts (Figure 1). A specific example on how the

microsimulation decides of a patient’s outcome given a set of independent

predictors on day 1 is provided as supplementary material (Tables E1-E4). If the

patient remained in the ICU to the next day, the simulation generated SOFA scores

for day 2 using the appropriate SOFA sub-models (Tables E5-E22). The probability

of being discharged alive from the ICU on day 2, remaining in the ICU to the next

day, and dying in the ICU on day 2 was recalculated and the fate of the patient

determined (Figure E1). This process was iterated until the patient was ICU

discharge or ICU day 30.

To assess model accuracy we calculated mean ICU mortality, mean ICU

length of stay, average daily organ-specific and global (sum of organ-specific scores

on any given day) SOFA scores, and cumulative (over the entire ICU stay) organ-

specific and global SOFA scores for the entire simulated cohort. The probability

distribution of the predictions was derived from running the microsimulation 500

times. We generated standardized ratios (SR) and 95% confidence intervals (CI) for

all evaluated outcomes and organ dysfunction scores using prediction from the

model as the numerator and actual observations in the development cohort as the

9


denominator.

Model validation

We validated the model in 460 patients from the remaining 12 ICUs using the

day 1 predictors to initiate the microsimulation. We used a random sample of ICUs

to increase the external validity of the model. Indeed, prediction models are

typically applied in situations where both patients and therapeutic approaches to

those patients vary from the development environment. Again, we simulated the

ICU time course of 460 patients selected with replacement from the validation

cohort and predicted the mean values for ICU mortality, ICU length of stay, and

daily and cumulative organ-specific and global SOFA scores. We compared the

mean values for each outcome predicted by the model to those observed in the

validation cohort and calculated SR and 95% CI.

Organ-organ interaction

To investigate organ-organ interaction we constructed standard linear

regression equations for SOFA scores for the entire cohort irrespective of time

period and examined the magnitude and significance of the regression coefficients

for predicting single organ SOFA scores of the next day based on SOFA scores of the

current day. The strength of interaction is conveyed by the magnitude of the

regression coefficients.

Statistical procedures

We compared proportions using Chi-square statistics. We compared lengths

of stay and organ failure scores using the Mann-Whitney U test. We assumed a

significance level of p<0.05 for all comparisons. We used the backwards stepwise

10


procedure to select significant predictors for the multinomial sub-models (p<.01).

We built the microsimulation model with the @RISK© 4.5.2 software (Palisade

Corporation, Newfield, NY, www.palisade.com).

11

http://www.palisade.com/


Results

Study population

The characteristics of the development and validation cohorts are provided in

Table 1. The development and validation cohorts had similar distributions

regarding age, sex, and location prior to the ICU admission. There were more

emergent surgical and fewer acute coronary patients in the development cohort.

The development cohort had less severe renal dysfunction over the time course of

the ICU stay. Otherwise, there was no difference in the cumulative global SOFA

between the cohorts. ICU mortality was similar in the development and validation

cohorts. Mean ICU length of stay was not significantly different between the

cohorts. Of the entire cohort, 544 (37.5%), 257 (17.7%), 128(8.8%) and 58 (4.0%)

patients stayed at least 7, 14, 21 or more than 30 days in the ICU, respectively.

The proportion of missing values was 11.0% overall, was higher for the

hepatic system (40.3%), and lower for the neurologic (10.9%), cardiovascular

(8.5%), pulmonary (3.1%), renal (1.7%) and coagulation systems (1.6%). There

were twice as many missing values in the second half of the ICU stay compared to

the first half (p<0.001) and an equal proportion of missing values in the

development and validation sets (p=0.84).

Model performance

We provide the coefficients of the multinomial equations derived to predict

ICU outcomes (death, discharged alive from the ICU, remained in ICU until next

day) and SOFA scores as supplementary data (Tables E1-E22).

Organ dysfunction

The model performed well in predicting the cumulative SOFA in the

12


development and validation cohorts for both single organ and global scores (Table

2). In the validation cohort, actual (5.5 [95% CI: 5.2-5.9]) and predicted mean daily

SOFA scores (5.1 [4.8-5.5]) were close (p=0.32). Actual and predicted global SOFA

score per patient were 48.2 (41.6-54.8) and 45.2 (39.8-50.6) (P=0.47). The ability

of the model to describe the time course of organ dysfunction is displayed for the

validation cohort in Figures 2. The model predicted sequential organ dysfunction

better in some organ systems than others, most noticeably underestimating renal

and hepatic dysfunction late in the ICU course.. Although the model appeared to

predict the general trends correctly, it did not reflect acute changes in mean SOFA

scores, as can be seen for the hematologic system scores (Figure 2). Age and type

of admission and ICU day were predictive of outcome. Sex was only predictive of

the evolution of the pulmonary score. Age was predictive of the evolution of all

scores except the hepatic and hematologic scores. Type of admission was

predictive of all scores except for the hematologic system. Finally, ICU day was a

significant predictor of the evolution for scores for all systems except the

pulmonary system.

ICU length of stay

In the development cohort, the model predicted that 18 patients, or 1.8%

(1.1%-2.6%), would still be in the ICU at the end of the 30-day study period and

thus underestimated the observed proportion of 3.9% by 21 patients.

Consequently, the model predicted 223 (2.9%) fewer ICU-days than observed

(7,576 days) in this cohort of 989 patients. However, the predicted and observed

mean ICU lengths of stay were not significantly different (7.5 vs. 7.7 days, p=0.14).

In the validation cohort, the model predicted that 3.1% of the validation cohort

(1.5%-4.7%) would still be in the ICU at the end of the 30-day period, while the

13


observed proportion of patients was 4.0% (Figure 3). The model underpredicted the

observed number of 3,766 ICU-days by 204 days, or 5.5%, in this cohort of 460

patients. The predicted and observed mean ICU lengths of stay were similar (7.7

vs. 8.2 days, p=0.30). Using different imputation methods for missing data

resulted in predicted ICU lengths of stays ranging from 7.4 to 7.8 days.

ICU mortality

The microsimulation predicted overall ICU mortality well in both the

development and validation cohorts (Table 2). The predicted and observed

mortalities for the development and validation cohorts were 21.0% (17.9%-24.9%)

and 21.0%, and 20.1% (16.2%-24.0%) and 20.0%, respectively. The Observed and

predicted survival curve for the validation cohort are very close (Figure 3). Using

different imputation methods for missing data resulted in predicted mortality

ranging from 19.8% to 21.7% in the validation cohort.

Organ-organ interaction

Organ-organ interaction is presented in Table 3. Not surprisingly, a given

organ SOFA score was most predictive of the SOFA score of that organ on the next

day. However, the model revealed many organ-organ interactions. SOFA score of

several organ systems were predictive of those of other organ system (e.g.,

cardiovascular, hematologic, and neurologic dysfunctions were all significant

predictors of the pulmonary SOFA score on the next day, while hepatic and renal

dysfunction were not [Table 3, first row]). As suggested by the magnitude of the

regression coefficients, there is strong two-way cardio-pulmonary interaction.

Cardiovascular dysfunction is associated with dysfunction in all other organ

systems, confirming its central role. Interestingly, the hematologic dysfunction

14


appears to be associated with subsequent hepatic dysfunction more than the

reverse.

15


Discussion

We constructed a dynamic microsimulation model to predict the temporal

patterns of multiple outcomes, such as ICU mortality, organ dysfunction, and ICU

length of stay in critically ill cohorts of patients based on demographic and clinical

characteristics on the day of ICU admission. The model was developed from

previously assembled data in a heterogeneous ICU population from 37 ICUs and

validated in patients from 12 other ICUs, where treatment, processes of care, and

ICU discharge policies might have varied widely compared to the development

cohort. The model predicted ICU mortality, average daily and cumulative amount

of organ failure, and ICU length of stay well.

Many authors have used organ failure scores as mortality prediction tools

[10, 12, 20, 21]. We have not considered this application because our purpose was

not to predict a particular outcome for an individual patient. Instead, we developed

a tool using a single modeling platform to predict the longitudinal time course of

multiple outcomes such as death, ICU discharge, and amount of organ failure in

cohorts of ICU patients with known characteristics on admission. Rangel-Frausto, et

al. recently developed a Markov model of the natural history and time course of

sepsis in the ICU population. These, authors presented an elegant Markov model of

the progression of sepsis, but did not report on organ dysfunction, nor did they

allow transitional probabilities to vary in time during the ICU stay [22]. Because we

wished to use organ dysfunction as the main predictor of outcome, it would have

been difficult to use a standard Markov paradigm[23], given the very large number

of states needed (one for each organ dysfunction combination, ICU discharge, and

death), the lack of sufficient data to generate time-dependent transition

probabilities requires to populate such a model.

16


Although previous reports described the prognostic importance of changes in

the levels of organ dysfunction, the current study is the first to describe direct

estimates of organ system dysfunction and interaction in a large cohort of ICU

patients, and to predict its course. Microsimulation is particularly well suited to

model systems that cannot be modeled with standard analytic techniques [16].

Because of the time-dependence of the SOFA scores, the problem of predicting

trajectories dynamical aspects of the problem remains the topic on ongoing

statistical research and is proving to be very difficult. Microsimulation has become

the method of choice to simulate the dynamics of complex systems with a large

number of configurations, the transitions between which vary in time. Applications

range from molecular processes to population dynamics.

The model provided several new insights into organ-organ interaction. The

level of dysfunction in any organ system was the most important predictor of the

level of dysfunction in the same organ on the following day. However, the logistic

equations identified several significant interactions, where the level of dysfunction

of a specific organ system on the following day was also determined by the

dysfunction of other organ systems. The model confirmed the central role of the

cardiovascular system in organ-organ interaction and the particular strength of the

cardio-pulmonary interaction. The current dataset is not sufficiently detailed to

allow a thorough exploration of such interactions, but our analysis suggests that

mechanisms can be hypothesized from analysis of observation data otherwise not

collected for that purpose.

There are a number of potential limitations of this study. The size of the

development set did not allow inclusion of potentially important predictors, such as

underlying disease or diagnosis on admission. The SOFA score may not reflect

17


organ dysfunction in a timely and accurate fashion. We constructed the model on a

heterogeneous ICU population, which may have compromised the predictive ability

of the model. Constructing the model on a more restricted case-mix (e.g., a cohort

of patients with sepsis) would be expected to improve predictability in this

population. Generalizability to other ICU populations could be limited. As ICU

discharge criteria may vary significantly from institution to institution, we

attempted to address this limitation by clustering the development and validation

sets at the ICU level. To maximize the amount of data available for modeling, we

imputed 11% of the organ failure scores using a scheme that seemed to

recapitulate existing physiology and clinical decision making. Stochastic methods

of imputation such as multiple imputation [19], although clearly more elegant and

appropriate in situations where missing data is random, may not be appropriate in

situations where data are missing because they were presumed known by treating

physicians. There is no standard way to assess the fit and validate microsimulation

models when actual data is not available. Fortunately, we could compare to

empiric observation. We therefore used a pragmatic approach to describe the

longitudinal time course of a patient cohort where no standard statistical

techniques are available to measure the closeness of a projected and observed

trajectory in time (e.g., describing a mean organ failure scores over the ICU stay).

We may have underestimated the uncertainty associated with the microsimulation.

We presented uncertainty around estimates originating from the stochasticity of the

microsimulation, but did not consider additional uncertainty associated with the

imperfect knowledge of model parameters themselves (such as regression

parameters). By retaining only highly significant parameters in the regression

equations we decreased this uncertainty, but did not eliminate it [16]. This level of

18


uncertainty relates to the epidemiological concept of precision of prediction. In

addition, there exists the possibility of the presence of a systematic bias embodied

in the structure of the model, the prediction equations, or less likely, because the

development population was fundamentally different from the validation population

in a way that we could not evaluate.

We conclude that dynamic microsimulation can forecast the temporal pattern

of multiple outcomes such as mortality, ICU discharge, burden of organ failure, and

resource use in heterogeneous cohorts of ICU patients from the profile of organ

dysfunction observed on admission. We suggest that such techniques may prove

practical as prediction tools that evaluate ICU performance on additional

dimensions besides the risk of death. Furthermore, such techniques could also

assist in staffing decisions, resource allocation, and the economic evaluation of ICU

specific interventions presumed to impact on organ failure.

19


References

1. Baue AE (1997) Multiple organ failure, multiple organ dysfunction syndrome, and

systemic inflammatory response syndrome. Why no magic bullets? Arch Surg 132:

703-707

2. Sauaia A, Moore FA, Moore EE, Norris JM, Lezotte DC, Hamman RF (1998) Multiple

organ failure can be predicted as early as 12 hours after injury. J Trauma 45: 291-301

3. Baue AE (1975) Multiple, progressive, or sequential systems failure. A syndrome of

the 1970s. Arch Surg 110: 779-781

4. Baue AE (1992) The horror autotoxicus and multiple-organ failure. Arch Surg 127:

1451-1462

5. (1992) American College of Chest Physicians/Society of Critical Care Medicine

Consensus Conference: Definitions for sepsis and organ failure and guidelines for the

use of innovative therapies in sepsis. Crit Care Med 20: 864-874

6. Sznajder M, Aegerter P, Launois R, Merliere Y, Guidet B, CubRea (2001) A cost-

effectiveness analysis of stays in intensive care units. Intensive Care Med 27: 146-

153

7. Knaus WA, Draper EA, Wagner DP, Zimmerman JE (1985) Prognosis in acute organ-

system failure. Ann Surg 202: 685-693

8. Vincent JL, Moreno R, Takala J, Willatts S, de Mendonca A, Bruining H , Reinhart CK,

Suter PM, Thijs LG (1996) The SOFA (Sepsis-related Organ Failure Assessment) score

to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-

Related Problems of the European Society of Intensive Care Medicine. Intensive Care

20


Med 22: 707-710

9. Le Gall JR, Klar J, Lemeshow S, Saulnier F, Alberti C, Artigas A, Teres D (1996) The

Logistic Organ Dysfunction system. A new way to assess organ dysfunction in the

intensive care unit. JAMA 276: 802-810

10. Hebert PC, Drummond AJ, Singer J, Bernard GR, Russell JA (1993) A simple multiple

system organ failure scoring system predicts mortality of patients who have sepsis

syndrome. Chest 104: 230-235

11. Wheeler A, Carmichael L, Christman B (1995) Renal function abnormalities in sepsis.

Am J Respir Crit Care Med 151: A317.

12. Ferreira FL, Bota DP, Bross A, Melot C, Vincent JL (2001) Serial evaluation of the SOFA

score to predict outcome in critically ill patients. JAMA 286: 1754-1758

13. Cox DR, Oakes D (1984) Analysis of Survival Data. Chapman & Hall, London

14. Gray RJ (1994) Spine-based tests in survival analysis. Biometrics 50: 640-652

15. Vincent JL, de Mendonca A, Cantraine F, Moreno R, Takala J, Suter PM, Sprung CL,

Colardyn F, Blecher S (1998) Use of the SOFA score to assess the incidence of organ

dysfunction/failure in intensive care units: Results of a multicenter, prospective study.

Crit Care Med 26: 1793-1800

16. Cronin KA, Legler JM, Etzioni RD (1998) Assessing uncertainty in microsimulation

modelling with application to cancer screening interventions. Stat Med 17: 2509-

2523

17. Moreno R, Vincent JL, Matos R, Mendonca A, Cantraine F, Thijs L, Takala J, Sprung C,

21


Antonelli M, Bruining H, Willatts S (1999) The use of maximum SOFA score to

quantify organ dysfunction/failure in intensive care. Results of a prospective,

multicentre study. Working Group on Sepsis related Problems of the ESICM. Intensive

Care Med 25: 686-696

18. Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, Sirio CA,

Murphy DJ, Lotring T, Damiano AM, Harrell FEJ (1991) The APACHE III prognostic

system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest

100: 1619-1636

19. Rubin DB (1987) Multiple Imputation for Nonresponse in Surveys. John Wiley & Sons,

New York, NY

20. Barie PS, Hydo LJ, Fischer E (1994) A prospective comparison of two multiple organ

dysfunction/failure scoring systems for prediction of mortality in critical surgical

illness. J Trauma 37: 660-666

21. Zimmerman JE, Knaus WA, Wagner DP, Sun X , Hakim RB, Nystrom PO (1996) A

comparison of risks and outcomes for patients with organ system failure: 1982-1990.

Crit Care Med 24: 1633-1641

22. Rangel-Frausto MS, Pittet D, Hwang T, Woolson RF, Wenzel RP (1998) The dynamics

of disease progression in sepsis: Markov modeling describing the natural history and

the likely impact of effective antisepsis agents. Clin Infect Dis 27: 185-190

23. Sonnenberg FA, Beck JR (1993) Markov models in medical decision making: A

practical guide. Med Decis Making 13: 322-338

22


Figure legends

Figure 1. Conceptual model of a patient’s progression through the time-course of an

ICU stay. The microsimulation starts on the day of ICU admission. Each day, a

patient may either be discharged alive from the ICU, remain in the ICU, or die in the

ICU. The probabilities of these events are derived from the development cohort and

based on the SOFA scores on that day. The SOFA scores of patients remaining in

the ICU are recalculated for the next day, based on the SOFA scores of the actual

day and demographic characteristics.

Figure 2. Predicted and observed sequential organ dysfunction in the validation cohort.

The predictions of mean daily SOFA scores (solid line) are based on 500

simulations. Observed mean daily scores are presented as 95% confidence

intervals (dashed lines) around the value of the mean. The model described well

the observed SOFA scores of the validation cohort. However, there was an

underestimation of the hepatic and renal scores for patients with longer ICU stays.

Figure 3. Predicted and observed outcomes in the validation cohort. The predictions

are based on 500 simulations. The model predicted mortality well in the validation

cohort, as both predicted mortality and ICU discharge curves (solid lines) and

observed curves (dashed lines) closely paralleled each other.

23


Table 1. Characteristics of the development and validation cohorts.

Development cohort (N = 989)

Validation cohort (N =

460)

P-value

Intensive care units 37 12Age (yrs ± SD) 53.8 ± 20.2 54.6 ± 19.4 0.51Males [N (%)] 621 (63.6) 288 (63.0) 0.52Admission source [N (%)] 0.38

Emergency room 346 (35.9) 174 (38.2) 0.31Hospital ward 256 (26.6) 114 (25.1) 0.55Operating room 255 (26.5) 121 (26.6) 0.96Other hospital 107 (11.1) 46 (10.1) 0.57

Admission type [N (%)] <0.001Elective surgery 180 (18.3) 80 (17.4) 0.71Emergent surgery 187 (19.0) 66 (14.4) 0.03Trauma 127 (12.9) 54 (11.8) 0.55Medical 460 (46.7) 213 (46.4) 0.93Acute coronary 32 (3.2) 46 (10.0) 0.01

Cumulative SOFA score*[mean ± SD (median)]

Respiratory 14.0 ± 18.4 (7) 13.7 ± 18.7, 7 0.75Cardiovascular 8.3 ± 14.5 (2) 7.8 ± 14.4, 2 0.59Neurologic 9.0 ± 17.6 (1) 9.1 ± 20.2, 0 0.93Renal 5.8 ± 14.7 (1) 8.2 ± 16.6, 2.5 0.005Hematologic 4.6 ± 8.6 (1) 4.8 ± 9.1, 1 0.68Hepatic 4.3 ± 10.6 (0) 4.6 ± 12.4, 0 0.65Global† 46.0 ± 63.5 (21) 48.2 ± 71.0, 23 0.87

ICU mortality (%) 21.0 21.1 0.76ICU length of stay [days ± SD (median)] 7.7 ± 7.5 ( 5) 8.2 ± 7.8, 5 0.29

24


Abbreviations: N=number of patients; SD=standard deviation; SOFA=sequential organ function assessment; ICU=intensive care unit.* Cumulative SOFA score is the sum of scores for an individual patient over the entire ICU stay.† Global SOFA score is the sum of single organ scores for an individual patient on a given day.

25


Table 2. Model performance in the development and validation cohorts.

Development cohort Validation cohortPredicte

dActual

SR (95% CI) Predicted

Actual

SR (95% CI)

Cumulative SOFA score* (mean)

Respiratory 13.7 14.0 0.98 (0.87-1.08)

13.7 13.7 1.00 (0.86-1.03)

Cardiovascular

8.1 8.3 0.97 (0.82-1.12)

8.5 7.8 1.08 (0.93-1.24)

Hematologic 4.9 4.5 1.08 (0.92-1.24)

5.3 4.6 1.15 (0.97-1.33)

Neurologic 8.5 8.9 0.95 (0.78-1.12)

7.9 9.2 0.86 (0.69-1.03)

Hepatic 3.0 4.3 0.70‡ (0.56-0.85)

3.4 4.6 0.74‡ (0.59-0.89)

Renal 5.3 5.8 0.92 (0.76-1.07)

6.5 8.3 0.78‡ (0.65-0.90)

Global† 43.4 45.9 0.95 (0.84-1.05)

45.2 48.4 1.04 (0.91-1.19)

ICU length of stay (days)

7.5 7.7 0.99 (0.91-1.08)

7.7 8.1 0.95 (0.86-1.03)

ICU mortality (%)

20.2 21.0 0.96 (0.78-1.14)

20.1 19.9 1.01 (0.81-1.21)

Abbreviations: SR=standardized ratio; CI=confidence interval; SOFA=sequential organ function assessment; ICU=intensive care unit.* Cumulative SOFA score is the sum of scores for an individual patient over the entire ICU stay.† Global SOFA score is the sum of single organ scores for an individual patient on a given day.‡ Significant difference between predicted and observed values (p<0.05).

26


Table 3. Organ-organ-interactions as evaluated by regression coefficients.*

Today’s score Yesterday’s score

PulmonaryCardio-vascular

Hematologic Neurologic Renal Hepatic

Pulmonary 0.707 0.078 0.038 0.032 NS NS

Cardiovascular 0.056 0.814 0.038 0.027 0.038 0.022

Hematologic NS 0.040 0.836 NS 0.019 0.026

Neurologic 0.017 0.027 NS 0.926 0.012 NS

Renal NS 0.037 0.034 NS 0.845 0.038

Hepatic 0.033 0.044 0.098 0.012 0.034 0.772

* The linear regression coefficients were derived from the entire set of observations and convey the strength of association between yesterday’s SOFA scores and today’s SOFA score across organ systems. A positive coefficient reflects a positive correlation (worse [better] scores yesterday correlate with worse [better] scores today), and a possible physiologic interaction.

NS=Not a significant predictor in the model (p<0.05) and therefore no associated coefficient in the final multinomial equations.

27

Figure 1

Development set37 ICUs

990 patients

SOFA dataset1449 patients

11417 ICU-days

Validation set12 ICUs

459 patients

Outcome sub-model•Daily probability of eachof 3 outcomes

SOFA sub-models•Daily probability of eachof 5 scores (0-4)•6 sub-models, 1 for each organ systems

Global modelIntegrate all submodels in a microsimulation that propels 500 cohorts of patients with known day 1 scores through ICU discharge

Apply global modelGenerate desired outcomes from day 1 SOFA scores

Compare to actual outcomes

Development set37 ICUs

990 patients

SOFA dataset1449 patients

11417 ICU-days

Validation set12 ICUs

459 patients

Outcome sub-model•Daily probability of eachof 3 outcomes

SOFA sub-models•Daily probability of eachof 5 scores (0-4)•6 sub-models, 1 for each organ systems

Global modelIntegrate all submodels in a microsimulation that propels 500 cohorts of patients with known day 1 scores through ICU discharge

Apply global modelGenerate desired outcomes from day 1 SOFA scores

Compare to actual outcomes

Figure 2

Respiratory

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 300

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30

Cardiovascular Hematologic

Neurologic Hepatic Renal

ICU day

Mea

n SO

FA s

core

Respiratory

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30

0

0.5

1

1.5

2

2.5

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30

0

0.5

1

1.5

2

2.5

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30

0

0.5

1

1.5

2

2.5

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30

0

0.5

1

1.5

2

2.5

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 300

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30

0

0.5

1

1.5

2

2.5

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30

0

0.5

1

1.5

2

2.5

0

0.5

1

1.5

2

2.5

0 5 10 15 20 25 30

Cardiovascular Hematologic

Neurologic Hepatic Renal

ICU day

Mea

n SO

FA s

core

Figure 3

0

0.2

0.4

0.6

0.8

1

3 6 9 12 15 18 21 24 27 30

ICU day

Cum

ulat

ive

prop

ortio

n

Predicted

Actual

Deceased

Still in ICU

Discharged from ICU

0

0.2

0.4

0.6

0.8

1

3 6 9 12 15 18 21 24 27 30

ICU day

Cum

ulat

ive

prop

ortio

n

Predicted

Actual

Deceased

Still in ICU

Discharged from ICU

microsimulation of organ dysfunction › ~cler › mscmp3780 › microsimulation_fina… · web...

Documents