A REVIEW ON DIFFERENT OPERATIVE RISK CALCULATORSDr Kitty Lo

Queen Elizabeth Hospital

CASE SCENARIO

88/M PMH: DM, HT, COPD, IHD, CRF E admitted with fever & severe abd pain BP 100/70 P125 Hb 8.5, WCC 22, Urea 18, Cr 250, K5.5 ABG pH 7.20, PaO2 9, HCO3 13 CT: Perforated CA caecum, no distant metastasis

WOULD YOU OPERATE ON THIS PATIENT?

WHY DO WE NEED OPERATIVE RISK CALCULATORS?

1. Guide decision making & informed consent and improve treatment planning

2. For surgical auditing & comparison of outcomes to improve quality of care

HOW TO DEFINE IF A PREDICTION MODEL IS ACCURATE?

Discrimination Calibration Observed: Estimated Ratio

DISCRIMINATION Ability of the model to assign higher probability of outcome to

patient who actually die than those who live Area under the receiver operative characteristic curve (c-

index) Value between 0.5 (random)-1.0 (perfect) Values 0.7-0.8: reasonable discrimination Values >0.8: good discrimination

CALIBRATION

Ability of the model to assign the correct probabilities of outcome to individual patients

Hosmer-Lemeshow χ² statistic The smaller the value, the better the

calibration

O:E MORTALITY RATIO

Ratio of observed no. of deaths to expected no. of deaths as calculated by the prediction model

In validation studies: 1.0 = perfect prediction

COMMONLY USED MODELS

RISK SCORING SYSTEMS

APACHE SAPS POSSUM & its variations ACS NSQIP

Dedicated for general surgery patients

APACHEAcute Physiology & Chronic Health Evaluation

APACHE- ACUTE PHYSIOLOGY & CHRONIC HEALTH EVALUATION

Developed for use in ICU setting APACHE II:

12 physiological parameters in the first 24h after hospital admission

Score modification available for surgical patients undergoing elective or emergency surgery

APACHE II Pros:

Reproducible & objective

Can be applied prospectively & retrospectively

Cons: Too complex for

general surgical use

Not all parameters are routinely measured

APACHE II

Validity: Extensively validated in ICU patients all over the

world1-3

Tend to overpredict mortality in surgical patients4

Trend to underpredict mortality in high risk cases and overpredict in low risk cases

1- Hariharan S, Moseley HSL, Kumar AY. Outcome evaluation in a surgical intensive care unit in Barbados, Anaesthesia. 2002; 57:434-4412- Beck DH, Taylor BL, Millar B, et al. Prediction of outcome from intensive care: a prospective cohort study comparing Acute Physiology and Chronic Health Evaluation II and III prognostic systems in a United Kingdom intensive care unit. Crit Care Med. 1997;25:9-15.3- Gianguiliani G, Mancini A, Gui D. Validation of a severity of illness score (APACHE II) in a surgical intensive care unit. Intensive Care Med, 1989; 15:519-522.4- Colpan A, Akinci E, Erbay A, et al. (2005). Evaluation of risk factors for mortality in intensive care units: a prospective study from a referral hospital in Turkey. Am J Infect Control 33:42-47.

SAPS IISimplified Acute Physiology Score II

SAPS II- SIMPLIFIED ACUTE PHYSIOLOGY SCORE

Variant of APACHE 17 variables Use data obtained

during the intensive care/ early post-op period

SAPS II

Validity: Extensively validated in ICUs over the world Not as commonly used as APACHE II Conflicting results

Some studies found SAPS II as a better predictor when compared with other scoring systems1-2

Other studies have found it to be less effective with relatively poor goodness-of-fit 3

1- Can MF, Yagci G, Tufan T, Ozturk E, et al. Can SAPS II predict operative mortality more accurately than POSSUM and P-POSSUM in patients with colorectal carcinoma undergoing resection? World J Surg. 2008; 32:589-595.2- Capuzzp M, Valpondi V, Sagarbi A, et al. Validation of severity scoring systems SAPSII and APACHE II in a single centre population. Intensive Care Med. 2000; 26:1779-1785.3- Sculier JP, Paesmans M, Markiewicz E, Berghmans T. Scoring systems in cancer patients admitted for an acute complication in a medical intensive care unit. Crit Care Med. 2000; 28:2786-2792.

POSSUMPhysiological and Operative Severity Score for the enUmeration of Mortality and morbidity

POSSUM- PHYSIOLOGICAL AND OPERATIVE SEVERITY SCORE FOR THE ENUMERATION OF MORTALITY AND MORBIDITY

Initially developed in 1991 for surgical audit 12 Physiological and 6 Operative factors Can predict both 30 day mortality and

morbidity

Copeland GP, Jones D, Walters M. POSSUM: a scoring system for surgical audit. Br J Surg. 1991; 78:355-360

POSSUM

Validity: Extensively studied and applied internationally

over the general surgical spectrum POSSUM mortality equation found to consistently

overpredict deaths; up to 7-fold in low risk patients 1-3

POSSUM was accurate in predicting post-op morbidity 1

1- Richards CH, Leitch FE, Horgan PG, McMillan DC. A systematic review of POSSUM and its related models as predictors of post-operative mortality and morbidity in patients undergoing surgery for colorectal cancer. J Gastrointest Surg 2010; 14:1511-1520.2- Whiteley MS, Prytherch DR, Higgins B, Weaver PC, Prout WG. An evaluation of the POSSUM surgical scoring system. Br J Surg. 1998 Sep; 85 (9):1217-203- POSSUM, p-POSSUM and Cr-POSSUM: Implementation issues in a United States health care system for prediction of outcome for colon cancer resection. Dis Colon Rectum 2004; 47:1435-1441

POSSUM

Inherent problems: The equation gives a minimum risk of death of

1.1%, far too high for minor procedures and fit patients

Physiological parameters are measured at the time of surgery, scores will vary depending on aggressiveness of resuscitation given pre-op

Operative severity scores includes blood loss and need for reoperation, which may be surgeon-dependent

Morbidity and complication is not easy to define

P-POSSUM – PORTSMOUTH POSSUM

A modification of the POSSUM system Uses the same physiological and operative

variables as POSSUM Recalibrated to a different regression

equation Only predicts in-hospital mortality

1- Prytherch DR, Whiteley MS, Higgins B, Weaver PC, Prout WG, Powell SJ. POSSUM and Portsmouth POSSUM for predicting mortality. Br J Surg. 1998; 85:1217-1220

P-POSSUM

Developed by analyzing 10000 surgical procedures 1

Using the first 2500 patients as a training set to produce the P-POSSUM predictor equation

Then applied prospectively to the remaining 7500 patients for validation

1- Prytherch DR, Whiteley MS, Higgins B, Weaver PC, Prout WG, Powell SJ. POSSUM and Portsmouth POSSUM for predicting mortality. Br J Surg. 1998; 85:1217-1220

CR-POSSUM- COLORECTAL POSSUM

Developed specifically for colorectal surgery Predicts in-hospital mortality Developed from data from almost 7000

patients

Tekkis PP, Prytherch DR, Kocher HM, et al. Development of a dedicated risk-adjustment scoring system for colorectal surgery (colorectal POSSUM). Br J Surg 2004;91:1174-1182

Tekkis PP, Prytherch DR, Kocher HM, et al. Development of a dedicated risk-adjustment scoring system for colorectal surgery (colorectal POSSUM). Br J Surg 2004;91:1174-1182

A SYSTEMATIC REVIEW OF POSSUM , P-POSSUM & CR-POSSUM IN COLORECTAL CANCER

19 studies 6929 patients To compare the predictive value of POSSUM,

P-POSSUM and CR-POSSUM in colorectal cancer surgery

Richards CH, Leitch FE, Horgan PG, McMillan DC. A systematic review of POSSUM and its related models as predictors of post-operative mortality and morbidity in patients undergoing surgery for colorectal cancer. J Gastrointest Surg 2010; 14:1511-1520.

SUMMARY DATA

1- Richards CH, Leitch FE, Horgan PG, McMillan DC. A systematic review of POSSUM and its related models as predictors of post-operative mortality and morbidity in patients undergoing surgery for colorectal cancer. J Gastrointest Surg 2010; 14:1511-1520.

P-POSSUM was most accurate model for predicting post-operative mortality after colorectal cancer surgery

POSSUM was accurate in predicting post-op complications

A SYSTEMATIC REVIEW OF POSSUM , P-POSSUM & CR-POSSUM IN COLORECTAL CANCER

A SYSTEMATIC REVIEW OF POSSUM , P-POSSUM & CR-POSSUM IN COLORECTAL CANCER

1- Richards CH, Leitch FE, Horgan PG, McMillan DC. A systematic review of POSSUM and its related models as predictors of post-operative mortality and morbidity in patients undergoing surgery for colorectal cancer. J Gastrointest Surg 2010; 14:1511-1520.

OTHER SPECIALTY-SPECIFIC POSSUM MODELS

O-POSSUM (oesophago-gastric) V-POSSUM (vascular)

ACS NSQIPAmerican College of Surgeons

National Surgical Quality Improvement Program

ACS NSQIP - AMERICAN COLLEGE OF SURGEONS NATIONAL SURGICAL QUALITY IMPROVEMENT PROGRAM

Data from 393 US hospitals, 1.4 million patients Developed a universal surgical risk estimation

tool Use 21 preoperative factors to predict 8

outcomes Mortality Morbidity Pneumonia Cardiac event Surgical site infection Urinary tract infection DVT Renal failureBilimoria KY, Liu YM, Paruch JL, et al. Development and evaluation of the Universal ACS NSQIP surgical risk

calculator: A decisional aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013; 217:833-842

ACS NSQIP

Validity: Excellent discrimination for mortality (c-statistic

0.944), morbidity (c-statistic 0.816) and the 6 additional complications (c-statistic >0.8)

Limitation: Does not account for the indication of the

procedure Application:

Calculated prospectively as a decision-support tool

Quality indicator for surgical auditing

Bilimoria KY, Liu YM, Paruch JL, et al. Development and evaluation of the Universal ACS NSQIP surgical risk calculator: A decisional aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013; 217:833-842

Cohen ME, Liu Yaoming, Ko CY, Hall BL. Improved surgical outcomes for ACS NSQIP hospitals over time. Evaluation of hospital cohorts with up to 8 years of participation. Annals of Surgery. 2015.

CASE SCENARIO

Proceeded to EOT with Right hemicolectomy performed

Post op not extubated to ICU, required mechanical ventilation & CVVH & inotropic support

Complicated by chest infection and AMI in early post op period

After a stormy initial period, finally made a full recovery and discharged home on day 30

PREDICTED RISK

CONCLUSION

No scoring system is perfect Serve as a guide to informed consent and

manage expectations Identify high risk patients preoperatively &

facilitate decision making regarding intensity of post-op monitoring

Role in surgical auditing

THANK YOU

WHAT IS A SURGICAL AUDIT?

A systematic appraisal of the implementation and outcome of any process in the context of prescribed targets and standards

Commonly assessed outcomes include 30 day or in-hospital mortality and morbidity

Crude morbidity & mortality affected by many factors

Account for the variation in the physiological condition of the patient and the severity of the procedure (Case-mix) for fair comparison

Allows for comparison of performance between individual hospitals and surgeons

APPLICATION OF ACS NSQIP

Annals of Surgery 2015

ACS NSQIP

Prediction models for mortality & morbidity recalibrated every 6 months

Each hospital given risk adjusted O/E ratios that permit comparison & for targeting quality improvement efforts

Cohen ME, Liu Yaoming, Ko CY, Hall BL. Improved surgical outcomes for ACS NSQIP hospitals over time. Evaluation of hospital cohorts with up to 8 years of participation. Annals of Surgery. 2015.

Improving performance over time

Estimate 0.8% reductions in mortality, 3.1% reductions in morbidity and 2.6% reductions in surgical site infection annually

The longer the duration of time in the program, the greater the magnitude of quality improvement

Cohen ME, Liu Yaoming, Ko CY, Hall BL. Improved surgical outcomes for ACS NSQIP hospitals over time. Evaluation of hospital cohorts with up to 8 years of participation. Annals of Surgery. 2015.

POSSUM MORBIDITY Haemorrhage

Wound and deep Infection

Chest, UTI, Deep, Septicaemia, PUO Wound dehiscence

Superficial and deep Anastomotic leak Thrombosis

DVT, PE, CVA, MI Cardiac failure Impaired renal function Hypotension Respiratory failure

WHAT IS PREREQUISITE FOR A GOOD MODEL?

Quick and easy to use Small number of variables Reproducible Applicable across the general surgical

spectrum, both elective & emergency Validated internationally by a large number

of centres