bmjopen.bmj.com · for peer review only 9 measure of predictive performance: as depicted in figure...

BMJ Open is committed to open peer review. As part of this commitment we make the peer review history of every article we publish publicly available. When an article is published we post the peer reviewers’ comments and the authors’ responses online. We also post the versions of the paper that were used during peer review. These are the versions that the peer review comments apply to. The versions of the paper that follow are the versions that were submitted during the peer review process. They are not the versions of record or the final published versions. They should not be cited or distributed as the published version of this manuscript. BMJ Open is an open access journal and the full, final, typeset and author-corrected version of record of the manuscript is available on our site with no access controls, subscription charges or pay-per-view fees (http://bmjopen.bmj.com). If you have any questions on BMJ Open’s open peer review process please email

[email protected]

on February 15, 2020 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2018-028302 on 29 July 2019. D

ownloaded from

http://bmjopen.bmj.com/

[email protected]


For peer review onlyRisk scores identifying elderly inpatients at risk of 30-day

unplanned readmission and Accident and Emergency Department visit: A systematic review

Journal: BMJ Open

Manuscript ID bmjopen-2018-028302

Article Type: Research

Date Submitted by the Author: 06-Dec-2018

Complete List of Authors: Schwab, Camille ; Hôpital Européen Georges Pompidou, Pharmacie; Institut Pierre Louis d'Epidemiologie et de Sante Publique, Equipe 6Hindlet, Patrick; Institut Pierre Louis d'Epidemiologie et de Sante Publique, Equipe 6Sabatier, Brigitte; Assistance Publique - Hopitaux de Paris, Pharmacy; INSERM UMR 1138, team 22, Cordelier Research CenterFernandez, Christine; Institut Pierre Louis d'Epidemiologie et de Sante Publique, Equipe 6; Hopital Saint-Antoine, PharmacieKorb-Savoldelli, Virginie; Hôpital Européen Georges Pompidou, Pharmacie; Universite Paris-Sud Faculte de Pharmacie de Chatenay-Malabry,

Keywords: PUBLIC HEALTH, Elderly, Readmission, Clinical risk scores

For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml

BMJ Open on F

ebruary 15, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2018-028302 on 29 July 2019. Dow

nloaded from


For peer review only

1

Risk scores identifying elderly inpatients at risk of 30-day unplanned readmission and Accident and Emergency Department visit: A systematic review

1,2Schwab C. (PharmD, PhD Student), 1,3,4Hindlet P. (PharmD, PhD), 2,5Sabatier B. (PharmD,

PhD), 1,3,4Fernandez C. (PhD, Pr), 2,4Korb-Savoldelli V.(PharmD, PhD)

Author information:1Sorbonne Universités, UPMC Univ Paris 06, UMR-S 1136, Institut Pierre Louis D’Epidémiologie et de Santé Publique, Paris, France2Service Pharmacie, Hôpital Européen Georges Pompidou, Assistance Publique – Hôpitaux de Paris, 75015 Paris, France3Service de Pharmacie, Hôpital Saint-Antoine, Assistance Publique – Hôpitaux de Paris,75012 Paris, France4Université Paris-Sud, Faculté de Pharmacie, 92296, Châtenay-Malabry, France5INSERM UMR 1138 Team 22: Information Sciences to Support Personalized Medicine, Cordelier Research Center, Paris, France

Corresponding Author: Dr Camille [email protected]ôpital Européen Georges Pompidou20-40, rue Leblanc75015 Paris, France.Tel: +331 56 09 51 49; fax: +331 56 09 25 59

Dr Patrick HINDLET: [email protected] Christine FERNANDEZ: [email protected] Brigitte SABATIER : [email protected] Virginie KORB-SAVOLDELLI: [email protected]

Key words: Decision support, elderly, readmission, adverse events, epidemiology and detectionWord count: 3 249 words

The authors have no conflicts of interest to declare

Page 1 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



2

ABSTRACT:Objectives: The aim of this systematic review was to describe and analyze the performance statistics of validated risk scores identifying elderly inpatients at risk of early unplanned readmission.Data sources: We identified potentially eligible studies by searching MEDLINE, EMBASE, COCHRANE and Web of Science. Our search was restricted to original studies, between 1966 and 2018. Study eligibility criteria: Original studies, which internally or externally validated the clinical scores of hospital readmissions in elderly inpatients.Participants: Elderly inpatients discharged home from hospital or returning home after an Accident and Emergency Department visit.Results: A total of 12 studies and five different scores were included in the review. The five scores present area under the receiving operating characteristic curve between 0.445 and 0.69. ISAR and TRST scores were the more frequently validated scores with ISAR being more sensitive and TRST more specific.Conclusions: The TRST and ISAR scores have been extensively studied and validated. The choice of the most suitable score relies on available patient data, patient characteristics and the foreseen clinical care intervention. In order to pair the intervention with the appropriate clinical score, further studies of external validation of clinical scores, identifying elderly patients at risk of early unplanned readmission, are needed.

Systematic review registration number: PROSPERO CRD42017054516

Article Summary - Strengths and limitations of this study:

Our systematic review includes clinical risk scores predicting elderly early unplanned readmission.Clinical risk scores give a numerical result and a cut-off, which might be more convenient for the current use.We identified potentially eligible studies by searching MEDLINE, EMBASE, COCHRANE and Web of Science. The data extraction grid is based on the STROBE Statement and the TRIPOD Statement.

Page 2 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



3

SYSTEMATIC REVIEW:

Introduction:

Hospital readmissions in the elderly are particularly deleterious, as they increase their risk of dependence and decompensation of comorbidities (1). Moreover, these readmissions are a burden for the health system organizations. Therefore, it has become a priority to decrease hospital readmission rates and for this purpose, unplanned hospital readmission, within 30 days after discharge, is now recognized as a valid quality indicator for hospital performance as these readmissions are likely to be connected to the index admission (2). Among all inpatients, the elderly are reportedly at higher risk of unplanned readmission: 17.3% of readmission for patients aged 85 years and older, versus 16.6% for patients under 64 years old (3). Thus, it is important to implement evidence-based clinical care interventions focused on reducing unplanned readmission of elderly patients (4). A time scale needs to be defined in order to measure potential benefits of such clinical care interventions. Moreover, this time limit has been introduced in guidelines or programs dealing with elderly hospital readmissions in USA and France (1,5).In order to better rationalize resources, healthcare authorities have stated that it is neither "necessary nor efficient to intervene for every patient”. Moreover, as mono-faceted interventions only have a limited effect (6), care bundle interventions should be implemented. However, these interventions are costly in terms of time, money and caregiving and there is a need to determine who will benefit from those interventions in priority (1).In this vein, clinical risk scores, which are prognosis tools estimating the probability or risk of future conditions (7), can help physicians and pharmacists to identify elderly patients at high risk of unplanned readmission within 30 days after discharge. Building a clinical risk score requires 3 successive steps (8): 1) the development; corresponding to the construction of the score on a derivation cohort, 2) an internal validation i.e. testing the score on the same population (validation cohort) and 3) an external validation, i.e. adjusting or updating the score in other patients. Internal and external validation studies evaluate the discrimination of the scores, using the c-statistic (9), and several metrological characteristics such as specificity or sensitivity. In our study, the c-statistic of risk scores that identify elderly inpatients at risk of 30 days unplanned readmission, will give the probability a randomly selected readmitted patient had a higher score than a non-readmitted patient. The specificity will give the ability to reject patients at low to moderate risk and sensitivity the ability to correctly include patients at high risk of readmission. In a recent systematic review, Kansagara et al. (10) listed validated readmission risk prediction models and showed that these risk prediction models are very heterogeneous: primary outcomes are readmissions ranging from 14 days to four years and populations can be either adults or elderly patients. Therefore, despite this work, this study has not enable us to select a clinical score suitable to elderly inpatients at risk of unplanned readmission within 30 days after discharge.

The aim of this systematic review was to describe and analyze the statistic performances of validated risk scores identifying elderly inpatients at risk of 30 days unplanned readmission.

Page 3 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



4

Methods:

This systematic review was registered in the PROSPERO database (registration number: CRD42017054516) and is presented according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Supplementary file 1) (11).

Eligibility criteria:We have systematically identified peer-reviewed original studies, with internally or externally validated clinical scores of unplanned hospital readmissions in elderly inpatients. We excluded reviews and studies identifying readmission risk factors, developing a statistical prediction model for readmission and clinical score not having been, previously, internally validated. Indeed, statistical prediction models give a probability of a patient being readmitted, whereas clinical risk scores give a number which can be compared to a threshold from which the patient is at high risk of readmission. Clinical scores are, thus, easier to use routinely.

We included studies:- in which participants were elderly inpatients (65 years old and over) discharged home after a

hospitalization or returning home after an Accident and Emergency Department (AED) visit. - in which the main outcome measured was unplanned readmission or return to the AED

within a month (28, 30 or 31 days) after discharge. We excluded studies that focused on specific conditions.

- where the score discrimination was evaluated using the c-statistic. The graphic representation of the c-statistic is the Receiving Operating Characteristic (ROC) curve. We excluded studies which had used neither the c-statistic, nor the AUC of the ROC curves for the validation.

Information sources and search strategy:We identified potentially eligible studies by searching in four databases MEDLINE EMBASE, COCHRANE and Web of Science. Our search was restricted to original human studies, published in French or English, between 1966 and 2017. The search strategies of the four databases were approved by a university librarian (Supplementary file 2). The electronic search was conducted on January 13th 2017 and updated on January 19th 2018. In addition to searching databases, we checked reference lists of selected studies and researched also the studies citing the selected studies, on Web of Science

Study selection:After identification, studies were screened independently by 2 reviewers (CS and VKS) on the

basis of title and abstract. Then, the full text was read for potential inclusion in the systematic review and agreement between the 2 reviewing authors (CS and VKS) was assessed using Cohen's κ statistic (12). A Cohen's κ coefficient comprised between 0.60 and 0.74 was considered as good, and very good if superior to 0.75. In the case of disagreement, a third reviewer (PH) was a priori designed to be consulted.

Page 4 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



5

Data collection process:We developed a data extraction grid based on The Strengthening the Reporting of

Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies (13) and the Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement (14). General data extracted from full-text included: authors, year of publication, design, setting, participants, data sources, study size, quantitative variables and statistical analysis methods. For each selected study, CS extracted the general data and completed the grid. Independently, VKS checked all the data collected in the grid. Disagreements were resolved by discussion and consensus.Using the same process, we extracted metrological data (AUC of the ROC curves, sensitivity, specificity, positive predictive value, negative predictive value, negative and positive likelihood ratio and accuracy) of the selected scores. Discrimination was considered fair for a c-statistic comprised between 0.5 and 0.69, and good when the c-statistic was superior to 0.7.

Quality assessment:Assessment of the risk of bias in individual studies was performed independently by 2

reviewers (CS and VKS), using the validated Effective Public Health Practice Project (EPHPP) quality assessment tool (15). This instrument, usable for any quantitative study design, was adapted to our study design in order to evaluate 5 types of bias: selection bias, study design, data collection method, statistical analysis and withdrawals and dropouts. The risk of bias was individually rated for each component (strong, moderate or weak risk) and added to assign a global rating to each study (strong, moderate or weak risk of bias).The risk of bias across studies was assessed using a graphic representation of the percentage of studies with a strong, moderate or weak risk of bias for each source of bias considered in the evaluation of the risk of bias in individual studies.

Patient and public involvement statement:No patient was involved in this systematic review.

Page 5 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



6

Results:

Result of search strategy:From the four databases, we extracted 2,484 references (Supplementary file 3). After

removal of 320 duplicates, we screened 2,164 studies on the basis of titles and abstracts leading to the exclusion of 2,075 irrelevant publications. A total of 93 publications were reviewed for eligibility. Nine studies finally met the inclusion criteria. We found three additional records by manual search (by checking the reference lists). Thus, a total of 12 studies (16–27) and five different scores were included in the present analysis. The degree of agreement between the two reviewers was good, with a κ coefficient of 0.72 (95%CI: 0.50-0.93).

Study characteristics:Study characteristics are presented in table I. The 12 included studies were published

between 2000 and 2017. Six were conducted in Europe (16–19,25,26), four in North America (USA and Canada)(20–22,27), one in Asia (Singapore)(23) and one in Australia (24). Design, population, interventions and outcome will be detailed below.

Page 6 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



7

Table I: Study characteristics

Data source Definition of readmissionAuthor

Year Country Scores Population Design Patient interview EMR AED

visitUnplanned

readmission Both

Internal validationsMc Cusker

2000 Canada ISAR AED patients, >65 years Prospective multicentric (n=4), + +

Meldon2003 USA TRST AED patients, >65 years Prospective multicentric

(n=2), + +

Arendts2015 Australia Nomogram AED patients, >65 years Prospective monocentric + + +

External validationsBraes2009 Belgium TRST, ISAR,

VIP hospitalized patients, >65 years Prospective monocentric, + +

Cotter2012

United-Kingdom LACE AED patients, geriatric inpatients NA, monocentric + +

Graf2012 Switzerland ISAR, TRST AED patients, >75 years Prospective monocentric + +

Lee2008 Canada TRST AED patients, >65 years Prospective multicentric

(n=3), + +

Moons2007 Belgium ISAR, TRST hospitalized and AED patients, >65

years Prospective monocentric, + +

Salvi2012 Italy ISAR, TRST hospitalized and AED patients, >65

years Prospective monocentric + +

Singler2014 Germany ISAR AED patients, >65 years Prospective monocentric + +

Suffoletto2016 USA

ISAR objective,

ISAR subjective

AED patients, >65 years Prospective multicentric (n=2) + +

Low2017 Singapore LACE hospitalized patients, >65 years Retrospective monocentric + +

AED = Accident and Emergency Department, EMR = Electronic Medical Record, ISAR = Identification of Senior At Risk, LACE = Lengths of stay, Admission, Charlson, Emergency, NA = Not Available, TRST = Triage Risk Screening Tool, USA = United States of America, VIP = Variable Indicative of Placement

Page 7 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



8

Study design: All the studies were prospective observational cohort studies, except one which was a retrospective observational cohort study (23) and one in which the design was not specified (26). Eight studies were monocentric (16–19,23–26) and four were multicentric (20–22,27).

Study population: Participants were inpatients, 65 years old and over in 10 studies (16–24,27) or 75 years old and over in one study (25). For one study, the age for inclusion was not mentioned, but patients were admitted to a geriatric ward (26). The mean age varies from 74 to 84.8 years old. Participants were discharged home after either an AED visit (16,18–22,24,25,27), a hospitalization (23,26) or after an AED visit followed by a hospital admission (17). The readmission rate varies from 12.1% to 28.4%

Intervention: Among the 12 studies, three internally validated a clinical score predicting hospital readmissions (21,22,24), meaning that included patients constituted validation cohorts. The other nine studies consisted of an external validation of scores, meaning that the scores were tested in a new cohort, different from the one used for validation. The validation cohort studies of these scores included adult inpatients and thus, they were not included in the present review.

Outcomes: The definition of readmission differs between studies: it was either an AED visit (16,18,21,24), an unplanned readmission (17,23,26) or both (composite outcome) (19,20,22,25,27) within one month (from 28 to 30 days). For the ISAR and TRST scores, the outcomes were an AED visit or a composite outcome, except for the Braes et al. study (17) where the outcome was an unplanned readmission. In this study, the c-statistic was lower than in the other studies (0.445 for the ISAR score and 0.478 for the TRST). Regarding the LACE index, the outcome was only an unplanned readmission (23,26).

Scores characteristics (Supplementary files 4, 5 and 6):The scores characteristics will be detailed bellow, by presenting predictors and measure of predictive performance.

Predictors: The median number of items included in the scores was 5 (range 3-8). As depicted in figure 1, the predictors composing the five scores can be divided into five categories: demographics, functional impairment, hospitalization, polymedication and comorbidity. Functional impairment and comorbidity are the most used predictors, whereas demographics are used in one score, the Nomogram. The assessment of each category differs between scores, eg the functional impairment can be measured with the capacity to walk (TRST), the ability to use a phone (VIP) or the presence of a home carer (Nomogram). Likewise, polymedication varies from more than 3 drugs to more than 6. All the categories are not use in every score, eg the LACE index assesses readmission risk with hospitalization characteristics (length of stay, admission type and history of AED visit) and comorbidity.Each predictor can be obtained at admission, except for the length of stay (LACE index) which will be obtained at discharge.

Measure of predictive performance: As depicted in figure 2, the c-statistics for the ISAR score were close and comprised between 0.608 and 0.690, except for Braes et al. study (17), whereas the c-statistics of the TRST score were scattered and comprised between 0.48 and 0.65. The TRST score

Page 8 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



9

had a better discrimination (AUC of the ROC curve of 0.647) in the internal validation study than in the external validation studies. The three other scores (VIP, Nomogram and LACE) have fair discriminations: the c-statistics were comprised between 0.5 and 0.7, the Nomogram having the best c-statistics (0.65). Moreover, the AUC of the ROC curves of the ISAR score were quite similar: 0.55 (IC95%: 0.49-0.61) and 0.595 (IC95%: 0.581-0.608), respectively.

As depicted in figure 3, TRST had a higher specificity and a lower sensitivity - the sensitivities varied from 0.62 to 0.871 and the specificities from 0.215 to 0.63 - whereas ISAR has a higher sensitivity and a lower specificity - the sensitivities varied from 0.79 to 0.918 and the specificities from 0.19 to 0.40 -, and they showed good reproducibility. The two other scores, LACE and VIP, with only one study for each, showed better sensitivity and specificity than ISAR and TRST but had been externally validated only once. Finally, the specificity and sensitivity of the Nomogram were not available.

In the external validation study by Braes et al. (17), the VIP score showed an AUC of the ROC curve of 0.516, a sensitivity of 0.22 and a specificity of 0.81 (Supplementary file 6). The others metrological qualities are presented in the supplementary files, but they were not calculated for each score (Supplementary files 4, 5 and 6).

Risk of bias within studies (table II):All included studies were observational studies. Thus, we evaluated whether the study was

prospective (low risk of bias) or retrospective (moderate risk of bias).All the studies were at low risk of bias (prospective studies), except Low et al. (23) which was a retrospective cohort study, and Cotter et al. (26), where the design was not specified.As we assume that multicentric studies have a more representative population than monocentric studies, we identified four studies with a low risk of bias (multicentric studies)(20–22,27) and eight studies with a moderate risk of bias (monocentric studies)(16–19,23–26).To determine if patients had an unplanned readmission, the data collection for these observational studies was made from the medical records (high risk of bias), with the patient (moderate risk of bias) or both or with a national clinical data base (low risk of bias). The medical records rarely indicated if an unplanned readmission occurred in a different hospital, because 20 or 30% of readmissions can be to a different hospital to the index admission. But the telephone interview and the national clinical data bases, they make it possible to overcome this classification bias.We then evaluated whether the withdrawals and drop-outs were reported in terms of number and/or reasons per group (low risk of bias) or not (high risk of bias). Four studies had not reported the withdrawals and drop-outs (19,23,24,27). We evaluated the statistical analysis as to whether the discrimination and the metrological data (low risk of bias), or only the discrimination (moderate risk of bias) were calculated.Finally, we rated the global risk of bias: three studies (16,17,22) had a global low risk of bias, seven (18–21,23,25,26) had a moderate risk of bias and two (23,24) had a global high risk of bias.

Page 9 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



10

Table II: Risk of bias within studies

Type of biasArendts

et al.2015

Braes et al.2009

Cotteret al.2012

Graf et al.2012

Leeet al.2008

Meldon et al.2033

Moonset al.2007

Salviet al.2012

Singleret al.2014

Suffolettoet al.2016

Mc Cusker

et al.2000

Lowet al.2017

Design NA

Setting

Data collection NA

Withdrawals and dropouts

Statistical analysis

Global rating

NA = Not Available, low risk of bias, moderate risk of bias, high risk of bias. Global rating: LOW = no high risk of bias, MODERATE = 1 high risk of bias, HIGH = 2 or more high risk of bias

Page 10 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



11

Risk of bias across studies:The global risk of bias, as presented in figure 4, was assessed through a graphical representation of risk percentages for each source of bias. It shows a low risk of bias for the design, the withdrawals and dropouts and the statistical analysis. The risk was moderate for the setting and high for the data collection.

Discussion

Summary of evidence:In this systematic review of the literature, we identified five validated risk clinical scores (ISAR, TRST, Nomogram, LACE and VIP) identifying elderly patients at risk of unplanned readmission within one month after discharge. These five scores showed only fair discrimination (c-statistic less than 0.7). Among these five scores, the ISAR and the TRST are the most common scores, having undergone several external validations. Those two scores have used electronic medical records, and have predicted unplanned readmissions and AED visits. In the included studies, presenting similar design, the predictive validities of the ISAR and TRST scores were constant, showing that these scores are reproducible.

We have considered scores identifying elderly patients at risk of early unplanned readmission. Some other clinical scores identify adult inpatients at risk of unplanned readmission. For example, the HOSPITAL score, a clinical score developed in the USA in 2013, was internationally externally validated in an adult population, including elderly (32) and has shown good performance with a c-statistic of 0.71. Unfortunately, it has not been specifically validated in an elderly population. Similarly, the LACE score was initially developed for adult inpatients (29). However, the LACE score showed, in the two external validation studies on an elderly population included in this systematic review (23,26), a lower discrimination (c-statistics of 0.55 and 0.595) than in the initial study (c-statistic = 0.7)(29). Thus, some clinical scores identifying patients at risk of unplanned readmission might be more effective on an adult population, rather than on an elderly population. In a recent study, Cooksley et al. (33) externally validated the LACE score in an adult inpatient population from two hospitals in Denmark. In this study, the authors evaluated the modification in discrimination of LACE score by age group and found that the discriminatory power of the LACE score decreased with increasing age. The authors also evaluated the change in discrimination of the HOSPITAL score, and came to the same conclusion: the discriminatory power of the HOSPITAL score decreases with increasing age.

We only included studies dealing with general elderly population. On the contrary, some clinical risk scores focus on specific condition. For example, Pack et al.(34) developed and validated a score predicting readmission after heart valve surgery and Echevarria et al. (35) described a score predicting readmission after hospitalization for acute exacerbation of chronic-obstructive pulmonary disease. These two scores predicted readmission with fair discrimination: c-statistic = 0.67 and 0.70, respectively. Echevarria et al. (35) also externally validated the LACE score for the prediction of readmission after hospitalization for acute exacerbation of chronic-obstructive pulmonary disease and found a c-statistic of 0.65. The LACE score was also externally validated to predict readmission in patients hospitalized with heart failure, and showed a fair discrimination, with a c-statistic of 0.59 (36). Hence, clinical scores focusing on specific patient populations or general risk scores used on

Page 11 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



12

specific patient population have not shown a better discrimination than general scores used on general populations.

For the ISAR and TRST scores, the c-statistics seem to be related to the outcome: the c-statistics appeared, indeed, lower for a prediction of an unplanned readmission, compared to an AED visit or a composite outcome.

Limitations:Our systematic review has some limitations. Unlike the review by Kansagara et al. (10), we excluded risk prediction models, in order to include only clinical risk scores which might be more convenient for the current use. Clinical risk scores indeed give a numerical result and a cut-off, which could be automatically calculated by the medical software. Moreover, we excluded validation studies on adult inpatients, where the outcome was a readmission within more than 1 month. For example, we excluded the HOSPITAL score (37) which has been externally validated in an international adult population, showing a good discrimination, with a c-statistic of 0.72 (95%CI, 0.72-0.72), or the 80+ score (38), an internally validated score on an elderly population, predicting the readmission within 1 year, with a good discrimination (c-statistic = 0.72 [95%CI, 0.66-0.77]).

Conclusions:This systematic review showed that several validated scores identifying elderly patients at risk of early readmission are available. Among all scores, ISAR and TRST scores have been extensively studied and validated and three promising scores (VIP, Nomogram and LACE) need further examination, even if the choice of the best fitting score to our patients depends on the available data, the patient and intervention characteristics. Their use can facilitate the implementation of bundle interventions on high risk patients, given the limited resources available.

Funding: This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectorsAcknowledgements: The authors would like to acknowledge Drs Agnes deChartres and Germain Perrin for their informed advice for the writing of this systematic review, Sophie Guiquerro, university librarian of the University of Paris Descartes for her support developing the literature search strategy and Stella Ghouti for the correction of the manuscript.

Author Contributions: Concept, design: C. Schwab, V. Savoldelli and P. Hindlet. Data Collection: C. Schwab and V. Savoldelli. Interpretation of results: C. Schwab, V. Savoldelli, P. Hindlet and C. Fernandez. Drafting of manuscript: C. Schwab. Critical revision of manuscript for important intellectual content: B. Sabatier, C. Schwab, V. Savoldelli, P. Hindlet and C. Fernandez.

Figure legends: Figure 1: Categories of predictors included in risk scoresFigure 2: c-statistics and sample sizes of included studiesFigure 3: Sensitivity, specificity and c-statistics of the scores ISAR, LACE, TRST and VIPFigure 4: Risk of bias across studiesCompeting interests: None declared.

Page 12 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



13

Patient consent: Not requiredData sharing statement: All available data are presented in the manuscript, figures, table and supplementary files.

References:

1. Haute Autorité de Santé. Comment réduire le risque de réhospitalisations évitables des personnes âgées ? 2013.

2. Fischer C, Lingsma HF, Marang-van de Mheen PJ, Kringos DS, Klazinga NS, Steyerberg EW. Is the Readmission Rate a Valid Quality Indicator? A Review of the Evidence. PLoS ONE [Internet]. 2014 Nov 7 [cited 2018 Feb 26];9(11). Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4224424/

3. Barnett ML, Hsu J, McWilliams JM. Patient Characteristics and Differences in Hospital Readmission Rates. JAMA Intern Med. 2015 Nov;175(11):1803–12.

4. Legrain S, Tubach F, Bonnet-Zamponi D, Lemaire A, Aquino J-P, Paillaud E, et al. A new multimodal geriatric discharge-planning intervention to prevent emergency visits and rehospitalizations of older adults: the optimization of medication in AGEd multicenter randomized controlled trial. J Am Geriatr Soc. 2011 Nov;59(11):2017–28.

5. McIlvennan CK, Eapen ZJ, Allen LA. Hospital Readmissions Reduction Program. Circulation. 2015 May 19;131(20):1796–803.

6. Hansen LO, Young RS, Hinami K, Leung A, Williams MV. Interventions to reduce 30-day rehospitalization: a systematic review. Ann Intern Med. 2011 Oct 18;155(8):520–8.

7. Moons KGM, Royston P, Vergouwe Y, Grobbee DE, Altman DG. Prognosis and prognostic research: what, why, and how? BMJ. 2009 Feb 23;338:b375.

8. Moons KGM, Kengne AP, Grobbee DE, Royston P, Vergouwe Y, Altman DG, et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart Br Card Soc. 2012 May;98(9):691–8.

9. Han K, Song K, Choi BW. How to Develop, Validate, and Compare Clinical Prediction Models Involving Radiological Parameters: Study Design and Statistical Methods. Korean J Radiol. 2016;17(3):339–50.

10. Kansagara D, Englander H, Salanitro A, Kagen D, Theobald C, Freeman M, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011 Oct 19;306(15):1688–98.

11. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009 Oct;62(10):e1-34.

12. Cohen J. A Coefficient of Agreement for Nominal Scales. Educ Psychol Meas. 1960 Jan 4;20(1):37–46.

13. von Elm E. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies. Ann Intern Med. 2007 Oct 16;147(8):573.

Page 13 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



14

14. Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015 Jan 7;350:g7594.

15. Thomas BH, Ciliska D, Dobbins M, Micucci S. A process for systematically reviewing the literature: providing the research evidence for public health nursing interventions. Worldviews Evid Based Nurs. 2004;1(3):176–84.

16. Moons P, De Ridder K, Geyskens K, Sabbe M, Braes T, Flamaing J, et al. Screening for risk of readmission of patients aged 65 years and above after discharge from the emergency department: predictive value of four instruments. Eur J Emerg Med Off J Eur Soc Emerg Med. 2007 Dec;14(6):315–23.

17. Braes T, Moons P, Lipkens P, Sterckx W, Sabbe M, Flamaing J, et al. Screening for risk of unplanned readmission in older patients admitted to hospital: predictive accuracy of three instruments. Aging Clin Exp Res. 2010 Aug;22(4):345–51.

18. Salvi F, Morichi V, Lorenzetti B, Rossi L, Spazzafumo L, Luzi R, et al. Risk stratification of older patients in the emergency department: comparison between the Identification of Seniors at Risk and Triage Risk Screening Tool. Rejuvenation Res. 2012 Jun;15(3):288–94.

19. Singler K, Heppner HJ, Skutetzky A, Sieber C, Christ M, Thiem U. Predictive validity of the identification of seniors at risk screening tool in a German emergency department setting. Gerontology. 2014;60(5):413–9.

20. Suffoletto B, Miller T, Shah R, Callaway C, Yealy DM. Predicting older adults who return to the hospital or die within 30 days of emergency department care using the ISAR tool: subjective versus objective risk factors. Emerg Med J EMJ. 2016 Jan;33(1):4–9.

21. McCusker J, Cardin S, Bellavance F, Belzile E. Return to the emergency department among elders: patterns and predictors. Acad Emerg Med Off J Soc Acad Emerg Med. 2000 Mar;7(3):249–59.

22. Meldon SW, Mion LC, Palmer RM, Drew BL, Connor JT, Lewicki LJ, et al. A brief risk-stratification tool to predict repeat emergency department visits and hospitalizations in older patients discharged from the emergency department. Acad Emerg Med Off J Soc Acad Emerg Med. 2003 Mar;10(3):224–32.

23. Low LL, Liu N, Ong MEH, Ng EY, Ho AFW, Thumboo J, et al. Performance of the LACE index to identify elderly patients at high risk for hospital readmission in Singapore. Medicine (Baltimore) [Internet]. 2017 May 12 [cited 2017 Dec 4];96(19). Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5428584/

24. Arendts G, Etherton-Beer C, Jones R, Bullow K, MacDonald E, Dumas S, et al. Use of a risk nomogram to predict emergency department reattendance in older people after discharge: a validation study. Intern Emerg Med. 2015 Jun;10(4):481–7.

25. Graf CE, Giannelli SV, Herrmann FR, Sarasin FP, Michel J-P, Zekry D, et al. Identification of older patients at risk of unplanned readmission after discharge from the emergency department - comparison of two screening tools. Swiss Med Wkly. 2012 Jan 3;141:w13327.

26. Cotter PE, Bhalla VK, Wallis SJ, Biram RWS. Predicting readmissions: poor performance of the LACE index in an older UK population. Age Ageing. 2012 Nov;41(6):784–9.

Page 14 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



15

27. Lee JS, Schwindt G, Langevin M, Moghabghab R, Alibhai SMH, Kiss A, et al. Validation of the triage risk stratification tool to identify older persons at risk for hospital admission and returning to the emergency department. J Am Geriatr Soc. 2008 Nov;56(11):2112–7.

28. McCusker J, Bellavance F, Cardin S, Trépanier S, Verdon J, Ardman O. Detection of older people at increased risk of adverse health outcomes after an emergency visit: the ISAR screening tool. J Am Geriatr Soc. 1999 Oct;47(10):1229–37.

29. van Walraven C, Dhalla IA, Bell C, Etchells E, Stiell IG, Zarnke K, et al. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. CMAJ Can Med Assoc J J Assoc Medicale Can. 2010 Apr 6;182(6):551–7.

30. Arendts G, Fitzhardinge S, Pronk K, Hutton M, Nagree Y, Donaldson M. Derivation of a nomogram to estimate probability of revisit in at-risk older adults discharged from the emergency department. Intern Emerg Med. 2013 Apr;8(3):249–54.

31. Vandewoude MFJ, Geerts C a. M, d’Hooghe AHM, Paridaens KMJ. [A screening tool to identify older people at risk of adverse health outcomes at the time of hospital admission]. Tijdschr Gerontol Geriatr. 2006 Oct;37(5):203–9.

32. Donzé J, Aujesky D, Williams D, Schnipper JL. Potentially avoidable 30-day hospital readmissions in medical patients: derivation and validation of a prediction model. JAMA Intern Med. 2013 Apr 22;173(8):632–8.

33. Cooksley T, Nanayakkara PWB, Nickel CH, Subbe CP, Kellett J, Kidney R, et al. Readmissions of medical patients: an external validation of two existing prediction scores. QJM Mon J Assoc Physicians. 2016 Apr;109(4):245–8.

34. Pack QR, Priya A, Lagu T, Pekow PS, Engelman R, Kent DM, et al. Development and Validation of a Predictive Model for Short- and Medium-Term Hospital Readmission Following Heart Valve Surgery. J Am Heart Assoc. 2016 Aug 31;5(9).

35. Echevarria C, Steer J, Heslop-Marshall K, Stenton SC, Hickey PM, Hughes R, et al. The PEARL score predicts 90-day readmission or death after hospitalisation for acute exacerbation of COPD. Thorax. 2017 Aug;72(8):686–93.

36. Yazdan-Ashoori P, Lee SF, Ibrahim Q, Van Spall HGC. Utility of the LACE index at the bedside in predicting 30-day readmission or death in patients hospitalized with heart failure. Am Heart J. 2016 Sep;179:51–8.

37. Donzé JD, Williams MV, Robinson EJ, Zimlichman E, Aujesky D, Vasilevskis EE, et al. International Validity of the HOSPITAL Score to Predict 30-Day Potentially Avoidable Hospital Readmissions. JAMA Intern Med. 2016 Apr;176(4):496–502.

38. Alassaad A, Melhus H, Hammarlund-Udenaes M, Bertilsson M, Gillespie U, Sundström J. A tool for prediction of risk of rehospitalisation and mortality in the hospitalised elderly: secondary analysis of clinical trial data. BMJ Open. 2015 Feb 18;5(2):e007259.

Page 15 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



280x184mm (96 x 96 DPI)

Page 16 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



338x211mm (300 x 300 DPI)

Page 17 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



338x211mm (300 x 300 DPI)

Page 18 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



338x211mm (300 x 300 DPI)

Page 19 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



SUPPLEMENTARY FILE 1: PRISMA CHECKLIST

Section/topic # Checklist item Reported on page #

TITLE

Title 1 Identify the report as a systematic review, meta-analysis, or both. 1

ABSTRACT

Structured summary 2 Provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; systematic review registration number.

2

INTRODUCTION

Rationale 3 Describe the rationale for the review in the context of what is already known. 3

Objectives 4 Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS).

3

METHODS

Protocol and registration 5 Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if available, provide registration information including registration number.

x

Eligibility criteria 6 Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years

considered, language, publication status) used as criteria for eligibility, giving rationale. 4

Information sources 7 Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched.

4

Search 8 Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated.

SUPPLEMENTARY FILE

Study selection 9 State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis).

4

Data collection process 10 Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators.

5

Data items 11 List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and simplifications made.

5

Page 20 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Risk of bias in individual studies

12 Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis.

5

Summary measures 13 State the principal summary measures (e.g., risk ratio, difference in means). SUPPLEMENTARY FILE

Synthesis of results 14 Describe the methods of handling data and combining results of studies, if done, including measures of

consistency (e.g., I2) for each meta-analysis.

5

Risk of bias across studies 15 Specify any assessment of risk of bias that may affect the cumulative evidence (e.g., publication bias, selective reporting within studies).

5

Additional analyses 16 Describe methods of additional analyses (e.g., sensitivity or subgroup analyses, meta-regression), if done, indicating which were pre-specified.

x

RESULTS

Study selection 17 Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram.

5

Study characteristics 18 For each study, present characteristics for which data were extracted (e.g., study size, PICOS, follow-up period) and provide the citations.

5

Risk of bias within studies 19 Present data on risk of bias of each study and, if available, any outcome level assessment (see item 12). 6

Results of individual studies

20 For all outcomes considered (benefits or harms), present, for each study: (a) simple summary data for each intervention group (b) effect estimates and confidence intervals, ideally with a forest plot.

7 & 8

Synthesis of results 21 Present the main results of the review. If meta-analyses are done, include for each, confidence intervals and measures of consistency.

Figure 2 & 3

Risk of bias across studies 22 Present results of any assessment of risk of bias across studies (see Item 15). 8

Additional analysis 23 Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses, meta-regression [see Item 16]).

x

Page 21 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




DISCUSSION

Summary of evidence 24 Summarize the main findings including the strength of evidence for each main outcome; consider their relevance to key groups (e.g., healthcare providers, users, and policy makers).

8

Limitations 25 Discuss limitations at study and outcome level (e.g., risk of bias), and at review-level (e.g., incomplete retrieval of identified research, reporting bias).

9

Conclusions 26 Provide a general interpretation of the results in the context of other evidence, and implications for future research.

9

FUNDING

Funding 27 Describe sources of funding for the systematic review and other support (e.g., supply of data); role of funders for the systematic review.

9

Page 22 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



SUPPLEMENTARY FILE 2: SEARCH STRATEGIES Database: EMBASE

# Searches Results

#1 'hospital readmission'/exp

#2 'hospital readmission'

#3 #1 OR #2

#4 'risk assessment'/exp

#5 'risk assessment'

#6 #4 OR #5

#7 (OR)

#8 'aged'

#9 'aged'/exp

#10 #8 OR #9

#11 #3 AND #6 AND #10 811

Database: Web of Science

# Searches Results

#1 TOPIC: (risk assessment) OR TOPIC: (score)

Indexes=SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH, BKCI-S, BKCI-

SSH, ESCI Timespan=All years

944,281

#2 TOPIC: (aged) OR TOPIC: (elderly) OR TOPIC: (aged, 80 and over) OR

TOPIC: (old*)



3,246,931

# 3 TOPIC: (readmission)



12,976

# 4 #1 AND #2 AND #3 1,164

Database: MEDLINE

# Searches Results

#1 ("Patient Readmission"[Mesh])

#2 ("Risk Assessment"[Mesh])

#3 (("Aged"[Mesh]) OR ("Aged, 80 and over"[Mesh]))

#4 #1 AND #2 AND #3 508

Page 23 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from

http://apps.webofknowledge.com.proxy.scd.u-psud.fr/summary.do;jsessionid=9FCE7331F135B82363E5037CE7E4BB5B?product=WOS&doc=1&qid=18&SID=W2mAOC4uKAu7D2CWchG&search_mode=GeneralSearch&update_back2search_link_param=yes



Database: Cochrane

# Searches

#1 MeSH descriptor: [Aged] explode all trees 1252

#2 MeSH descriptor: [Aged, 80 and over] explode all trees 145

#3 #1 or #2 1252

#4 MeSH descriptor: [Patient Readmission] explode all trees 1092

#5 MeSH descriptor: [Risk Assessment] explode all trees 9803

#6 #3 and #4 and #5 1

Page 24 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



SUPPLEMENTARY FILE 3: FLOW DIAGRAM

SC

RE

EN

ING

E

LIG

IBIL

ITY

IN

CLU

SIO

N

IDE

NTIF

ICA

TIO

N Number of records identified through database searching: n = 2,484

Medline: n = 508, Web of Science: n = 1,164, Embase: n = 811, Cochrane: n = 1

Number of records screened on title and abstract: n = 2,164

Number of records excluded: n = 320

• Duplicates = 320

Number of records excluded: n = 2,075

• Off topic articles: n = 2,075

Number of full-text articles assessed for eligibility: n = 93


• Patients under 65 years old: n = 34

• No internal or external validation of a score:

n = 27

• Non-original studies: n = 7

• Specific disease: n = 7

• Readmission to more than 30 days: n = 8

• Admission after being admitted to a nursing

home: n = 1

Number of records included after full-text assessment: n = 9

Number of records added after hand search: n = 3

Total number of studies included in the qualitative synthesis: n = 12

Page 25 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



SUPPLEMENTARY FILE 4: RESULTS FOR THE ISAR SCORE

The ISAR screening tool is a six-question self-report questionnaire on dependence, history of hospitalization, cognition, vision and polypharmacy. It was

developed in Canada, in 1999 (28). The ISAR score was internally validated in the Mc Cusker et al. study (21), and externally validated in six of the twelve

included studies (16–20,25).

Author Year

Age m (sd or

IQR)

Number of

patients

Women n (%)

Readmission rates n (%)

c-statistic / AUC (ROC)

Sensitivity Specificity NPV PPV LR-N LR-P Accuracy

(%)

Mc Cusker* 2000

1.122 640

(57%) 216 (19.3%)

0.68 (0.61-0.75)

N N N N N N N

Braes 2009

76 (71-80)

213 106

(49.8%) 28 (14.7%) 0.445

0.60

0.30 0.81 0.12 1.37 0.84 34.0

Graf 2012

83.9 (5.7) 345 217

(62.9%) 85 (24.6%)

0.612 (0.55-0.68)

0.918 0.219

0.891 0.278 N N N

Moons 2007

74 (70-79.5)

83 45

(54.2%) 12 (15.8%) 0.608

0.79

0.37 0.89 0.22 0.59 1.24 44

Salvi 2012

81.7 (7.7) 2.057 1.233 (60%)

340 (17%) 0.63

(0.60-0.66) 0.8

0.36 0.9 0.21 N N N

Singler 2014

82.8 (5) 520 313

(60.2%) 83 (16%)

0.621 (0.57-0.67)

0.89 (0.84-0.92)

0.25 (0.20-0.30)

N N N N N

Suffoletto 2016

77(8) 202 112

(55%) 45 (22%)

0.66 (0.59-0.72)

0.91 (0.78-0.97)

0.19 (0.13-0.26)

N N 0.47 1.12 N

Suffoletto 2016

77(8) 202 112

(55%) 45 (22%)

0.69 (0.59-0.77)

0.87 (0.77-0.99)

0.40 (0.32-0.47)

N N 0 1.11 N

AUC(ROC): Area Under the Curve of the Receiver Operating Characteristic curve, NPV: Negative Predictive Value, PPV: Positive Predictive Value, LR-N:

Likelihood Ratio-Negative, LR-P: Likelihood Ratio-Positive, N: Item not reported in the publication.*Internal validation study

Page 26 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



SUPPLEMENTARY FILE 5: RESULTS FOR THE TRST SCORE

The TRST score is a five-item score developed in the United-States of America, in 2003 (22). It was designed for rapid administration by triage nurses in the

AED in order to identify elderly at risk of AED revisits or readmission. The five variables are: history of cognitive impairment, difficulty walking/transferring or

recent falls, taking five or more medications, AED use in previous 30 days or hospitalization in previous 90 days. In their study, Meldon et al. internally

validated the TRST score (22). This score was externally validated in five of the twelve included studies (16–18,25,27).

Authors Year

Age m (sd or

IQR)

Number of

patients

Women n (%)




(%)

Meldon* 2003

74.4 (6.9) 647 385 (59%) 142 (26%) 0.647 0.64 0.63 N N N N N

Braes 2009

76 (71-80)

213 106

(49.8%) 28 (14.7%) 0.478 0.56 0.4 0.84 0.13 1.11 0.93 42.3

Graf 2012

83.9 (5.7) 345 217

(62.9%) 85 (24.6%)

0.607 (0.54-0.67)

0.871 0.215 0.836 0.266 N N N

Lee 2008

76.6 (65-101)

788 459

(58.6%) 147 (18.7%)

0.61 (0.56-0.66)

0.62 (0.54-0.70)

0.57 (0.53-0.61)

N N N N N

Moons 2007

74 (70-79.5)

83 45 (54.2%) 12 (15.8%) 0.568 0.64 0.47 0.85 0.21 0.76 1.21 50

Salvi 2012

81.7 (70-79.5)

2.057 1.233 (60%)

340 (17%) 0.61

(0.58-0.65) 0.76 0.4 0.89 0.21 N N N


Likelihood Ratio-Negative, LR-P: Likelihood Ratio-Positive, N: Item not reported in the publication. *Internal validation study

Page 27 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



SUPPLEMENTARY FILE 6: RESULTS OF THE OTHER SCORES

The Nomogram (24) was developed in Australia, in 2013 (30) in an elderly population and predicts revisit to the AED within 28 days after discharge.

LACE score (Lengths of hospitalization stay (L), Acuity of the admission (A), Comorbidities of patients (C) and AED admissions (E)). It was developed in

Canada, in 2010, to identify adult inpatients at risk of death or urgent readmission within 30 days (29).

The Variable Indicative of Placement (VIP) was developed in an elderly Belgian population, in 2006 (31).

Authors Year Scores

Age m (sd

or IQR)

Number of

patients

Women n (%)



Sensitivity Specificity NPV PPV LR- N

LR- P

Accur- acy (%)

Arendts

2015 Nomo-gram

79 (72-85)

1.143 629

(55%) 325 (28.4%) 0.65 N N N N N N N

Braes 2009

VIP 76

(71-80)

213 106

(49.8%) 28 (14.7%) 0.516 0.22 0.81 0.86 0.16 0.96 1.17 72.6

Cotter 2012

LACE 84.8 (6.5)

507 N 90 (17.8%) 0.55

(0.49-0.61) N N N N N N N

Low 2017

LACE 74

(69-80)

17.006 9.118

(53.6%) 2.051

(12.1%)

0.595 (0.581-0.608)

0.069 (0.058-0.080)

0.963 (0.960-0.966)

0.883 (0.878-0.888)

0.020 (0.017-0.232

N N N

AUC(ROC): Area Under the Curv83e of the Receiver Operating Characteristic curve, NPV: Negative Predictive Value, PPV: Positive Predictive Value, LR-N:

Likelihood Ratio-Negative, LR-P: Likelihood Ratio-Positive, N: Item not reported in the publication. *No internal validation studies for those scores.

Page 28 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



PRISMA CHECKLIST


TITLE


ABSTRACT


2

INTRODUCTION



3

METHODS


x




4


SUPPLEMENTARY FILE


4


5


5

Page 29 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from






5




5


5


x

RESULTS


5


5




7 & 8


Figure 2 & 3



x

Page 30 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




DISCUSSION


8


9


9

FUNDING


9

Page 31 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Journal: BMJ Open

Manuscript ID bmjopen-2018-028302.R1


Date Submitted by the Author: 04-Apr-2019


<b>Primary Subject Heading</b>: Geriatric medicine

Secondary Subject Heading: Public health, Epidemiology



BMJ Open on F


http://bmjopen.bm

j.com/

BM


jopen-2018-028302 on 29 July 2019. Dow

nloaded from



1









Page 1 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



2

ABSTRACT:Objectives: The aim of this systematic review was to describe and analyze the performance statistics of validated risk scores identifying elderly inpatients at risk of early unplanned readmission.Data sources: We identified potentially eligible studies by searching MEDLINE, EMBASE, COCHRANE and Web of Science. Our search was restricted to original studies, between 1966 and 2018. Study eligibility criteria: Original studies, which internally or externally validated the clinical scores of hospital readmissions in elderly inpatients.Participants: Elderly inpatients discharged home from hospital or returning home after an Accident and Emergency Department visit.Results: A total of 12 studies and five different scores were included in the review. The five scores present area under the receiving operating characteristic curve between 0.445 and 0.69. ISAR and TRST scores were the more frequently validated scores with ISAR being more sensitive and TRST more specific.Conclusions: The TRST and ISAR scores have been extensively studied and validated. The choice of the most suitable score relies on available patient data, patient characteristics and the foreseen clinical care intervention. In order to pair the intervention with the appropriate clinical score, further studies of external validation of clinical scores, identifying elderly patients at risk of early unplanned readmission, are needed.




Page 2 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



3

SYSTEMATIC REVIEW:

Introduction:

Hospital readmissions in the elderly are particularly deleterious, as they increase their risk of dependence and decompensation of comorbidities (1). Moreover, these readmissions are a burden for the health system organizations. Therefore, it has become a priority to decrease hospital readmission rates and for this purpose, unplanned hospital readmission, within 30 days after discharge, is now recognized as a valid quality indicator for hospital performance as these readmissions are likely to be connected to the index admission (2). Among all inpatients, the elderly are reportedly at higher risk of unplanned readmission: 17.3% of readmission for patients aged 85 years and older, versus 16.6% for patients under 64 years old (3). Thus, it is important to implement evidence-based clinical care interventions focused on reducing unplanned readmission of elderly patients (4). A time scale needs to be defined in order to measure potential benefits of such clinical care interventions. Moreover, this time limit has been introduced in guidelines or programs dealing with elderly hospital readmissions in USA and France (1,5).In order to better rationalize resources, healthcare authorities have stated that it is neither "necessary nor efficient to intervene for every patient”. Moreover, as mono-faceted interventions only have a limited effect (6), care bundle interventions should be implemented. However, these interventions are costly in terms of time, money and caregiving and there is a need to determine who will benefit from those interventions in priority (1).In this vein, clinical risk scores, which are prognosis tools estimating the probability or risk of future conditions (7), can help physicians and pharmacists to identify elderly patients at high risk of unplanned readmission within 30 days after discharge. Building a clinical risk score requires 3 successive steps (8): 1) the development; corresponding to the construction of the score on a derivation cohort, 2) an internal validation i.e. testing the score on the same population (validation cohort) and 3) an external validation, i.e. adjusting or updating the score in other patients. Internal and external validation studies evaluate the discrimination of the scores, using the c-statistic (9), and several metrological characteristics such as specificity or sensitivity. In our study, the c-statistic of risk scores that identify elderly inpatients at risk of 30 days unplanned readmission, will give the probability a randomly selected readmitted patient had a higher score than a non-readmitted patient. The specificity will give the ability to reject patients at low to moderate risk and sensitivity the ability to correctly include patients at high risk of readmission. The Kansagara et al. (10) systematic review and its updated version from Zhou et al. (11) listed validated readmission risk prediction models and showed that these risk prediction models are very heterogeneous: primary outcomes are readmissions ranging from 14 days to four years and populations can be either adults or elderly patients. Therefore, despite these works, this study has not enable us to select a clinical score suitable to elderly inpatients at risk of unplanned readmission within 30 days after discharge.

The aim of this systematic review was to describe and analyze the statistic performances of validated risk scores identifying elderly inpatients at risk of 30 days unplanned readmission.

Page 3 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



4

Methods:










Page 4 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



5



Quality assessment:Assessment of the risk of bias in individual studies was performed independently by 2

reviewers (CS and VKS), using the validated Effective Public Health Practice Project (EPHPP) quality assessment tool (16). This instrument, usable for any quantitative study design, was adapted to our study design in order to evaluate 5 types of bias: selection bias, study design, data collection method, statistical analysis and withdrawals and dropouts. The risk of bias was individually rated for each component (strong, moderate or weak risk) and added to assign a global rating to each study. The global rating followed the EPHPP guidelines: a strong global rating means there is no weak rating, a moderate global rating means there is one weak rating and a weak global rating means there are two or more weak ratings.The risk of bias across studies was assessed using a graphic representation of the percentage of studies with a strong, moderate or weak risk of bias for each source of bias considered in the evaluation of the risk of bias in individual studies.


Page 5 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



6

Results:





Page 6 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



7




visitUnplanned

readmission Both




(n=2), + +




Cotter2012




(n=3), + +






Suffoletto2016 USA

ISAR objective,

ISAR subjective




Page 7 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



8



Intervention: Among the 12 studies, three internally validated a clinical score predicting hospital readmissions (22,23,25), meaning that included patients constituted validation cohorts. The other nine studies consisted of an external validation of scores, meaning that the scores were tested in a new cohort, different from the one used for validation. The validation cohort studies of these scores included adult inpatients and thus, they were not included in the present review.


Because of the large heterogeneity of the studies (data sources and definition of readmission, table I), a meta-analysis was not possible.

Scores characteristics:The scores characteristics will be detailed bellow, by presenting predictors and measure of predictive performance (Supplementary files 4, 5 and 6).

Predictors: The median number of items included in the scores was 5 (range 3-8). As depicted in figure 1, the predictors composing the five scores can be divided into five categories: demographics, functional impairment, hospitalization, polymedication and comorbidity. Functional impairment and comorbidity are the most used predictors, whereas demographics are used in one score, the Nomogram. The assessment of each category differs between scores, eg the functional impairment can be measured with the capacity to walk (TRST), the ability to use a phone (VIP) or the presence of a home carer (Nomogram). Likewise, polymedication varies from more than 3 drugs to more than 6. All the categories are not use in every score, eg the LACE index assesses readmission risk with hospitalization characteristics (length of stay, admission type and history of AED visit) and comorbidity.Each predictor can be obtained at admission, except for the length of stay (LACE index) which will be obtained at discharge.

Page 8 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



9

Measure of predictive performance: As depicted in figure 2, the c-statistics for the ISAR score were close and comprised between 0.608 and 0.690, except for Braes et al. study (18), whereas the c-statistics of the TRST score were scattered and comprised between 0.48 and 0.65. The TRST score had a better discrimination (AUC of the ROC curve of 0.647) in the internal validation study than in the external validation studies. The three other scores (VIP, Nomogram and LACE) have fair discriminations: the c-statistics were comprised between 0.5 and 0.65, the Nomogram having the best c-statistics (0.65). Moreover, the AUC of the ROC curves of the ISAR score were quite similar: 0.55 (IC95%: 0.49-0.61) and 0.595 (IC95%: 0.581-0.608), respectively.



The sensitivities and specificities were evaluated based on thresholds defined in the studies. Different thresholds could be defined to achieve different results for these two statistics.

Risk of bias within studies:All included studies were observational studies. Thus, we evaluated whether the study was

prospective (low risk of bias) or retrospective (moderate risk of bias).All the studies were at low risk of bias (prospective studies), except Low et al. (24) which was a retrospective cohort study, and Cotter et al. (27), where the design was not specified.As we assume that multicentric studies have a more representative population than monocentric studies, we identified four studies with a low risk of bias (multicentric studies)(21–23,28) and eight studies with a moderate risk of bias (monocentric studies)(17–20,24–27).To determine if patients had an unplanned readmission, the data collection for these observational studies was made from the medical records (high risk of bias), with the patient (moderate risk of bias) or both or with a national clinical data base (low risk of bias). The medical records rarely indicated if an unplanned readmission occurred in a different hospital, because about 20 % of readmissions can be to a different hospital to the index admission (29). But the telephone interview and the national clinical data bases, they make it possible to overcome this classification bias.We then evaluated whether the withdrawals and drop-outs were reported in terms of number and/or reasons per group (low risk of bias) or not (high risk of bias). Four studies had not reported the withdrawals and drop-outs (20,24,25,28). We evaluated the statistical analysis as to whether the discrimination and the metrological data (low risk of bias), or only the discrimination (moderate risk of bias) were calculated.Finally, we rated the global risk of bias: three studies (17,18,23) had a global low risk of bias, seven (19–22,24,26,27) had a moderate risk of bias and two (24,25) had a global high risk of bias (table II).

Page 9 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



10


Type of biasArendts

et al.2015

Braes et al.2009

Cotteret al.2012

Graf et al.2012

Leeet al.2008

Meldon et al.2033

Moonset al.2007

Salviet al.2012

Singleret al.2014


Mc Cusker

et al.2000

Lowet al.2017

Design NA

Setting

Data collection NA



Global rating


Page 10 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



11

Risk of bias across studies:The global risk of bias, as presented in figure 4, was assessed through a graphical representation of risk percentages for each source of bias. It shows a low risk of bias for the design, the withdrawals and dropouts and the statistical analysis. The risk was moderate for the setting and high for the data collection.

Discussion




Page 11 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



12



Limitations:Our systematic review has some limitations. Unlike the review by Kansagara et al. (10), we excluded risk prediction models, in order to include only clinical risk scores which might be more convenient for the current use. Clinical risk scores indeed give a numerical result and a cut-off, which could be automatically calculated by the medical software. Moreover, we excluded validation studies on adult inpatients, which implies that studies with subanalyses that examine a score among the elderly population while the primary analyses were performed for a broad population were excluded. This is the case, for example, of the study of Cooksley et al. (32), which have shown that the discriminatory power of the LACE index and the HOSPITAL score (30) decreased with increasing age. Similarly, we excluded validation studies that focus on specific conditions or where the outcome was readmission within more than 1 month. These exclusion criteria have limited the number of scores presented in this review, whereas some scores have shown good discrimination ability, and thus might be interesting to externally validate in other conditions. For example, the 80+ score (36), an internally validated score on an elderly population which focuses on the patients ‘prescription, predicts the readmission within 1 year, with a good discrimination (c-statistic = 0.72 [95%CI, 0.66-0.77]).




Page 12 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



13

Figure legends: Figure 1: Categories of predictors included in risk scoresFigure 2: c-statistics and sample sizes of included studiesFigure 3: Sensitivity, specificity and c-statistics of the scores ISAR, LACE, TRST and VIPFigure 4: Risk of bias across studiesCompeting interests: None declared.Patient consent: Not requiredData sharing statement: All available data are presented in the manuscript, figures, table and supplementary files.

References:











11. Zhou H, Della PR, Roberts P, Goh L, Dhaliwal SS. Utility of models to predict 28-day or 30-day unplanned hospital readmissions: an updated systematic review. BMJ Open. 2016 27;6(6):e011060.

Page 13 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



14














Page 14 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



15





29. Nasir K, Lin Z, Bueno H, Normand S-LT, Drye EE, Keenan PS, et al. Is same-hospital readmission rate a good surrogate for all-hospital readmission rate? Med Care. 2010 May;48(5):477–81.








Page 15 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



280x184mm (96 x 96 DPI)

Page 16 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



338x211mm (300 x 300 DPI)

Page 17 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



338x211mm (300 x 300 DPI)

Page 18 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



338x211mm (300 x 300 DPI)

Page 19 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from





TITLE


ABSTRACT


2

INTRODUCTION



3

METHODS


x




4


SUPPLEMENTARY FILE


4


5


5

Page 20 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from






5




5


5


x

RESULTS


5


5




7 & 8


Figure 2 & 3



x

Page 21 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




DISCUSSION


8


9


9

FUNDING


9

Page 22 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




# Searches Results



#3 #1 OR #2



#6 #4 OR #5

#7 (OR)

#8 'aged'

#9 'aged'/exp

#10 #8 OR #9

#11 #3 AND #6 AND #10 811


# Searches Results




944,281


TOPIC: (old*)



3,246,931




12,976

# 4 #1 AND #2 AND #3 1,164

Database: MEDLINE

# Searches Results




#4 #1 AND #2 AND #3 508

Page 23 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Database: Cochrane

# Searches



#3 #1 or #2 1252



#6 #3 and #4 and #5 1

Page 24 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




SC

RE

EN

ING

E

LIG

IBIL

ITY

IN

CLU

SIO

N

IDE

NTIF

ICA

TIO












n = 27





home: n = 1




Page 25 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from







Author Year

Age m (sd or

IQR)

Number of

patients

Women n (%)




(%)

Mc Cusker* 2000

1.122 640

(57%) 216 (19.3%)

0.68 (0.61-0.75)

N N N N N N N

Braes 2009

76 (71-80)

213 106

(49.8%) 28 (14.7%) 0.445

0.60

0.30 0.81 0.12 1.37 0.84 34.0

Graf 2012

83.9 (5.7) 345 217

(62.9%) 85 (24.6%)

0.612 (0.55-0.68)

0.918 0.219

0.891 0.278 N N N

Moons 2007

74 (70-79.5)

83 45

(54.2%) 12 (15.8%) 0.608

0.79

0.37 0.89 0.22 0.59 1.24 44

Salvi 2012

81.7 (7.7) 2.057 1.233 (60%)

340 (17%) 0.63

(0.60-0.66) 0.8

0.36 0.9 0.21 N N N

Singler 2014

82.8 (5) 520 313

(60.2%) 83 (16%)

0.621 (0.57-0.67)

0.89 (0.84-0.92)

0.25 (0.20-0.30)

N N N N N

Suffoletto 2016

77(8) 202 112

(55%) 45 (22%)

0.66 (0.59-0.72)

0.91 (0.78-0.97)

0.19 (0.13-0.26)

N N 0.47 1.12 N

Suffoletto 2016

77(8) 202 112

(55%) 45 (22%)

0.69 (0.59-0.77)

0.87 (0.77-0.99)

0.40 (0.32-0.47)

N N 0 1.11 N



Page 26 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from








Authors Year

Age m (sd or

IQR)

Number of

patients

Women n (%)




(%)

Meldon* 2003

74.4 (6.9) 647 385 (59%) 142 (26%) 0.647 0.64 0.63 N N N N N

Braes 2009

76 (71-80)

213 106

(49.8%) 28 (14.7%) 0.478 0.56 0.4 0.84 0.13 1.11 0.93 42.3

Graf 2012

83.9 (5.7) 345 217

(62.9%) 85 (24.6%)

0.607 (0.54-0.67)

0.871 0.215 0.836 0.266 N N N

Lee 2008

76.6 (65-101)

788 459

(58.6%) 147 (18.7%)

0.61 (0.56-0.66)

0.62 (0.54-0.70)

0.57 (0.53-0.61)

N N N N N

Moons 2007

74 (70-79.5)

83 45 (54.2%) 12 (15.8%) 0.568 0.64 0.47 0.85 0.21 0.76 1.21 50

Salvi 2012

81.7 (70-79.5)

2.057 1.233 (60%)

340 (17%) 0.61

(0.58-0.65) 0.76 0.4 0.89 0.21 N N N



Page 27 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from








Authors Year Scores

Age m (sd

or IQR)

Number of

patients

Women n (%)




LR- P

Accur- acy (%)

Arendts

2015 Nomo-gram

79 (72-85)

1.143 629

(55%) 325 (28.4%) 0.65 N N N N N N N

Braes 2009

VIP 76

(71-80)

213 106

(49.8%) 28 (14.7%) 0.516 0.22 0.81 0.86 0.16 0.96 1.17 72.6

Cotter 2012

LACE 84.8 (6.5)

507 N 90 (17.8%) 0.55

(0.49-0.61) N N N N N N N

Low 2017

LACE 74

(69-80)

17.006 9.118

(53.6%) 2.051

(12.1%)

0.595 (0.581-0.608)

0.069 (0.058-0.080)

0.963 (0.960-0.966)

0.883 (0.878-0.888)

0.020 (0.017-0.232

N N N



Page 28 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



PRISMA CHECKLIST


TITLE


ABSTRACT


2

INTRODUCTION



3

METHODS


x




4


SUPPLEMENTARY FILE


4


5


5

Page 29 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from






5




5


5


x

RESULTS


5


5




7 & 8


Figure 2 & 3



x

Page 30 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




DISCUSSION


8


9


9

FUNDING


9

Page 31 of 31


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Journal: BMJ Open

Manuscript ID bmjopen-2018-028302.R2


Date Submitted by the Author: 19-Jun-2019


<b>Primary Subject Heading</b>: Geriatric medicine

Secondary Subject Heading: Public health, Epidemiology



BMJ Open on F


http://bmjopen.bm

j.com/

BM


jopen-2018-028302 on 29 July 2019. Dow

nloaded from



1









Page 1 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



2

ABSTRACT:Objectives: The aim of this systematic review was to describe and analyze the performance statistics of validated risk scores identifying elderly inpatients at risk of early unplanned readmission.Data sources: We identified potentially eligible studies by searching MEDLINE, EMBASE, COCHRANE and Web of Science. Our search was restricted to original studies, between 1966 and 2018. Eligibility criteria: Original studies, which internally or externally validated the clinical scores of hospital readmissions in elderly inpatients.Data extraction and synthesis: A data extraction grid based on STROBE and TRIPOD Statements was developed and completed by two reviewers to collect general data. The same process was used to extract metrological data of the selected scores.Quality assessment of the included studies: Assessment of the quality and risk of bias in individual studies was performed by 2 reviewers, using the validated Effective Public Health Practice Project (EPHPP) quality assessment tool.Participants: Elderly inpatients discharged home from hospital or returning home after an accident and emergency Department visit.Results: A total of 12 studies and five different scores were included in the review. The five scores present area under the receiving operating characteristic (ROC)curve between 0.445 and 0.69. ISAR and TRST scores were the more frequently validated scores with ISAR being more sensitive and TRST more specific.Conclusions: The TRST and ISAR scores have been extensively studied and validated. The choice of the most suitable score relies on available patient data, patient characteristics and the foreseen clinical care intervention. In order to pair the intervention with the appropriate clinical score, further studies of external validation of clinical scores, identifying elderly patients at risk of early unplanned readmission, are needed.




Page 2 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



3

SYSTEMATIC REVIEW:

Introduction:

Hospital readmissions in the elderly are particularly deleterious, as they increase their risk of dependence and decompensation of comorbidities (1). Moreover, these readmissions are a burden for the health system organizations. Therefore, it has become a priority to decrease hospital readmission rates and for this purpose, unplanned hospital readmission, within 30 days after discharge, is now recognized as a valid quality indicator for hospital performance as these readmissions are likely to be connected to the index admission (2). Among all inpatients, the elderly are reportedly at higher risk of unplanned readmission: 17.3% of readmission for patients aged 85 years and older, versus 16.6% for patients under 64 years old (3). Thus, it is important to implement evidence-based clinical care interventions focused on reducing unplanned readmission of elderly patients (4). A time scale needs to be defined in order to measure potential benefits of such clinical care interventions. Moreover, this time limit has been introduced in guidelines or programs dealing with elderly hospital readmissions in USA and France (1,5).In order to better rationalize resources, healthcare authorities have stated that it is neither "necessary nor efficient to intervene for every patient”. Moreover, as mono-faceted interventions only have a limited effect (6), care bundle interventions should be implemented. However, these interventions are costly in terms of time, money and caregiving and there is a need to determine who will benefit from those interventions in priority (1).In this vein, clinical risk scores, which are prognosis tools estimating the probability or risk of future conditions (7), can help physicians and pharmacists to identify elderly patients at high risk of unplanned readmission within 30 days after discharge. Building a clinical risk score requires 3 successive steps (8): 1) the development; corresponding to the construction of the score on a derivation cohort, 2) an internal validation i.e. testing the score on the same population (validation cohort) and 3) an external validation, i.e. adjusting or updating the score in other patients. Internal and external validation studies evaluate the discrimination of the scores, using the c-statistic (9), and several metrological characteristics such as specificity or sensitivity. In our study, the c-statistic of risk scores that identify elderly inpatients at risk of 30-day unplanned readmission, will give the probability a randomly selected readmitted patient had a higher score than a non-readmitted patient. The specificity will give the ability to reject patients at low to moderate risk and sensitivity the ability to correctly include patients at high risk of readmission. In 2011, Kansagara et al. (10) conducted a systematic review to identify risk prediction models for hospital readmission. This systematic review included thirty studies. Among the thirty studies, fourteen chose unplanned readmission between 31 days and four years after discharge as outcome, six studies focused on specific conditions and seven focused on adult inpatients. We chose to exclude this kind of studies and to focus on elderly patients and all-cause unplanned readmissions within thirty days after discharge. The three remaining studies presented prediction models with no calculation of discrimination. We chose to include only studies for which the expression of discrimination was made by calculating the c-statistic or the area under the ROC curves. In 2016, while updating the review from Kansagara, Zhou et al. (11) included a total of 60 studies in their systematic review. Among the 60 included studies, only four focused on elderly patients and all-cause unplanned readmissions within thirty days after discharge, two were prediction models and one was a clinical risk score without calculation of the discrimination ability.

Page 3 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



4

Therefore, these studies have not enabled us to select a clinical score suitable for elderly inpatients at risk of unplanned readmission within 30 days after discharge.

The aim of this systematic review was to describe and analyze the statistical performances of validated risk scores identifying elderly inpatients at risk of 30-day unplanned readmission.

Methods:








Page 4 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



5





Quality assessment:Assessment of quality in individual studies was performed independently by 2 reviewers (CS

and VKS), using the validated Effective Public Health Practice Project (EPHPP) quality assessment tool (16). This instrument, usable for any quantitative study design, was adapted to our study design in order to evaluate the quality through 5 sections: selection bias, study design, data collection method, statistical analysis and withdrawals and dropouts. The quality was individually rated for each component (strong, moderate or weak) and added to assign a global rating to each study. The global rating followed the EPHPP guidelines: a strong global rating means there is no weak rating, a moderate global rating means there is one weak rating and a weak global rating means there are two or more weak ratings.The quality across studies was assessed using a graphic representation of the percentage of studies with a strong, moderate or weak rating for each sections considered in the evaluation of the quality in individual studies.


Page 5 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



6

Results:





Page 6 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



7




visitUnplanned

readmission Both




(n=2), + +




Cotter2012




(n=3), + +






Suffoletto2016 USA

ISAR objective,

ISAR subjective




Page 7 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



8



Intervention: Among the 12 studies, three internally validated a clinical score predicting hospital readmissions (22,23,25), meaning that included patients constituted validation cohorts. The other nine studies consisted of an external score validation, meaning that the scores were tested in a new cohort, different from the one used for validation. The validation cohort studies of these scores included adult inpatients and thus, they were not included in the present review.


Because of the large heterogeneity of the studies (data sources and definition of readmission, table I), a meta-analysis was not possible.

Scores characteristics:The scores characteristics will be detailed bellow, by presenting predictors and measure of predictive performance (Supplementary files 4, 5 and 6).

Predictors: The median number of items included in the scores was 5 (range 3-8). As depicted in figure 1, the predictors composing the five scores can be divided into five categories: demographics, functional impairment, hospitalization, polymedication and comorbidity. Functional impairment and comorbidity are the most used predictors, whereas demographics are used in one score (Nomogram). The assessment of each category differs between scores, eg the functional impairment can be measured with the capacity to walk (TRST), the ability to use a phone (VIP) or the presence of a home carer (Nomogram). Likewise, polymedication varies from more than 3 drugs to more than 6. All the categories are not use in every score, eg the LACE index assesses readmission risk with hospitalization characteristics (length of stay, admission type and history of AED visit) and comorbidity.Each predictor can be obtained at admission, except for the length of stay (LACE index) which will be obtained at discharge.

Page 8 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



9

Measure of predictive performance: As depicted in figure 2, the c-statistics for the ISAR score were close and comprised between 0.608 and 0.690, except for Braes et al. study (18), whereas the c-statistics of the TRST score were scattered and comprised between 0.48 and 0.65. The TRST score had a better discrimination (AUC of the ROC curve of 0.647) in the internal validation study than in the external validation studies. The three other scores (VIP, Nomogram and LACE) have fair discriminations: the c-statistics were comprised between 0.5 and 0.65, the Nomogram having the best c-statistics (0.65). Moreover, the AUC of the ROC curves of the ISAR score were quite similar: 0.55 (IC95%: 0.49-0.61) and 0.595 (IC95%: 0.581-0.608), respectively.



The sensitivities and specificities were evaluated based on thresholds defined in the studies. Different thresholds could be defined to achieve different results for these two statistics.

Quality assessment within studies:All included studies were observational studies. Thus, we evaluated whether the study was

prospective (low risk of bias) or retrospective (moderate risk of bias).All the studies were at low risk of bias (prospective studies), except Low et al. (24) which was a retrospective cohort study, and Cotter et al. (27), where the design was not specified.As we assume that multicentric studies have a more representative population than monocentric studies, we identified four studies with a low risk of bias (multicentric studies)(21–23,28) and eight studies with a moderate risk of bias (monocentric studies)(17–20,24–27).To determine if patients had an unplanned readmission, the data collection for these observational studies was made from the medical records (high risk of bias), with the patient (moderate risk of bias) or both or with a national clinical data base (low risk of bias). The medical records rarely indicated if an unplanned readmission occurred in a different hospital, because about 20 % of readmissions can be to a different hospital to the index admission (29). But the telephone interview and the national clinical data bases, they make it possible to overcome this classification bias.We then evaluated whether the withdrawals and drop-outs were reported in terms of number and/or reasons per group (low risk of bias) or not (high risk of bias). Four studies had not reported the withdrawals and drop-outs (20,24,25,28). We evaluated the quality of the statistical analysis as to whether the discrimination and the metrological data (low risk of bias), or only the discrimination (moderate risk of bias) were calculated.Finally, we rated the global quality: three studies (17,18,23) had a global low risk of bias, seven (19–22,24,26,27) had a moderate risk of bias and two (24,25) had a global high risk of bias (table II).

Page 9 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



10


Type of biasArendts

et al.2015

Braes et al.2009

Cotteret al.2012

Graf et al.2012

Leeet al.2008

Meldon et al.2033

Moonset al.2007

Salviet al.2012

Singleret al.2014


Mc Cusker

et al.2000

Lowet al.2017

Design NA

Setting

Data collection NA



Global rating


Page 10 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



11

Quality assessment across studies:The global quality assessment, as presented in figure 4, was assessed through a graphical representation of risk percentages for each source of bias. It shows a low risk of bias for the design, the withdrawals and dropouts and the statistical analysis. The risk was moderate for the setting and high for the data collection.

Discussion




Page 11 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



12



Limitations:Our systematic review has some limitations. Unlike the review by Kansagara et al. (10), we excluded risk prediction models, in order to include only clinical risk scores which might be more convenient for the current use. Clinical risk scores indeed give a numerical result and a cut-off, which could be automatically calculated by the medical software. Moreover, we excluded validation studies on adult inpatients, which implies that studies with subanalyses that examine a score among the elderly population while the primary analyses were performed for a broad population were excluded. This is the case, for example, of the study of Cooksley et al. (32), which have shown that the discriminatory power of the LACE index and the HOSPITAL score (30) decreased with increasing age. Similarly, we excluded validation studies that focus on specific conditions or where the outcome was readmission within more than 1 month. These exclusion criteria have limited the number of scores presented in this review, whereas some scores have shown good discrimination ability, and thus might be interesting to externally validate in other conditions. For example, the 80+ score (36), an internally validated score on an elderly population which focuses on the patients ‘prescription, predicts the readmission within 1 year, with a good discrimination (c-statistic = 0.72 [95%CI, 0.66-0.77]).




Page 12 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



13

Figure legends: Figure 1: Categories of predictors included in risk scoresFigure 2: c-statistics and sample sizes of included studiesFigure 3: Sensitivity, specificity and c-statistics of the scores ISAR, LACE, TRST and VIPFigure 4: Risk of bias across studiesCompeting interests: None declared.Patient consent: Not requiredData sharing statement: All available data are presented in the manuscript, figures, table and supplementary files.

References:











11. Zhou H, Della PR, Roberts P, Goh L, Dhaliwal SS. Utility of models to predict 28-day or 30-day unplanned hospital readmissions: an updated systematic review. BMJ Open. 2016 27;6(6):e011060.

Page 13 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



14














Page 14 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



15





29. Nasir K, Lin Z, Bueno H, Normand S-LT, Drye EE, Keenan PS, et al. Is same-hospital readmission rate a good surrogate for all-hospital readmission rate? Med Care. 2010 May;48(5):477–81.








Page 15 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



280x184mm (96 x 96 DPI)

Page 16 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



338x211mm (300 x 300 DPI)

Page 17 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



338x211mm (300 x 300 DPI)

Page 18 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from



338x211mm (300 x 300 DPI)

Page 19 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from





TITLE


ABSTRACT


2

INTRODUCTION



3

METHODS


x




4


SUPPLEMENTARY FILE


4


5


5

Page 20 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from






5




5


5


x

RESULTS


5


5




7 & 8


Figure 2 & 3



x

Page 21 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




DISCUSSION


8


9


9

FUNDING


9

Page 22 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




# Searches Results



#3 #1 OR #2



#6 #4 OR #5

#7 (OR)

#8 'aged'

#9 'aged'/exp

#10 #8 OR #9

#11 #3 AND #6 AND #10 811


# Searches Results




944,281


TOPIC: (old*)



3,246,931




12,976

# 4 #1 AND #2 AND #3 1,164

Database: MEDLINE

# Searches Results




#4 #1 AND #2 AND #3 508

Page 23 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




Database: Cochrane

# Searches



#3 #1 or #2 1252



#6 #3 and #4 and #5 1

Page 24 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from




SC

RE

EN

ING

E

LIG

IBIL

ITY

IN

CLU

SIO

N

IDE

NTIF

ICA

TIO












n = 27





home: n = 1




Page 25 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from







Author Year

Age m (sd or

IQR)

Number of

patients

Women n (%)




(%)

Mc Cusker* 2000

1.122 640

(57%) 216 (19.3%)

0.68 (0.61-0.75)

N N N N N N N

Braes 2009

76 (71-80)

213 106

(49.8%) 28 (14.7%) 0.445

0.60

0.30 0.81 0.12 1.37 0.84 34.0

Graf 2012

83.9 (5.7) 345 217

(62.9%) 85 (24.6%)

0.612 (0.55-0.68)

0.918 0.219

0.891 0.278 N N N

Moons 2007

74 (70-79.5)

83 45

(54.2%) 12 (15.8%) 0.608

0.79

0.37 0.89 0.22 0.59 1.24 44

Salvi 2012

81.7 (7.7) 2.057 1.233 (60%)

340 (17%) 0.63

(0.60-0.66) 0.8

0.36 0.9 0.21 N N N

Singler 2014

82.8 (5) 520 313

(60.2%) 83 (16%)

0.621 (0.57-0.67)

0.89 (0.84-0.92)

0.25 (0.20-0.30)

N N N N N

Suffoletto 2016

77(8) 202 112

(55%) 45 (22%)

0.66 (0.59-0.72)

0.91 (0.78-0.97)

0.19 (0.13-0.26)

N N 0.47 1.12 N

Suffoletto 2016

77(8) 202 112

(55%) 45 (22%)

0.69 (0.59-0.77)

0.87 (0.77-0.99)

0.40 (0.32-0.47)

N N 0 1.11 N



Page 26 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from








Authors Year

Age m (sd or

IQR)

Number of

patients

Women n (%)




(%)

Meldon* 2003

74.4 (6.9) 647 385 (59%) 142 (26%) 0.647 0.64 0.63 N N N N N

Braes 2009

76 (71-80)

213 106

(49.8%) 28 (14.7%) 0.478 0.56 0.4 0.84 0.13 1.11 0.93 42.3

Graf 2012

83.9 (5.7) 345 217

(62.9%) 85 (24.6%)

0.607 (0.54-0.67)

0.871 0.215 0.836 0.266 N N N

Lee 2008

76.6 (65-101)

788 459

(58.6%) 147 (18.7%)

0.61 (0.56-0.66)

0.62 (0.54-0.70)

0.57 (0.53-0.61)

N N N N N

Moons 2007

74 (70-79.5)

83 45 (54.2%) 12 (15.8%) 0.568 0.64 0.47 0.85 0.21 0.76 1.21 50

Salvi 2012

81.7 (70-79.5)

2.057 1.233 (60%)

340 (17%) 0.61

(0.58-0.65) 0.76 0.4 0.89 0.21 N N N



Page 27 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from








Authors Year Scores

Age m (sd

or IQR)

Number of

patients

Women n (%)




LR- P

Accur- acy (%)

Arendts

2015 Nomo-gram

79 (72-85)

1.143 629

(55%) 325 (28.4%) 0.65 N N N N N N N

Braes 2009

VIP 76

(71-80)

213 106

(49.8%) 28 (14.7%) 0.516 0.22 0.81 0.86 0.16 0.96 1.17 72.6

Cotter 2012

LACE 84.8 (6.5)

507 N 90 (17.8%) 0.55

(0.49-0.61) N N N N N N N

Low 2017

LACE 74

(69-80)

17.006 9.118

(53.6%) 2.051

(12.1%)

0.595 (0.581-0.608)

0.069 (0.058-0.080)

0.963 (0.960-0.966)

0.883 (0.878-0.888)

0.020 (0.017-0.232

N N N



Page 28 of 28


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


ownloaded from


bmjopen.bmj.com · for peer review only 9 measure of predictive performance: as depicted in figure...

Documents