caring for hospital patients with covid-19 quality of care

Supported by:

Caring for hospital patients with COVID-19Quality of care in England examined by case record review

Published August 2021

Caring for hospital patients with COVID-19

© Royal College of Physicians 2021 2

The Royal College of Physicians

The Royal College of Physicians (RCP) plays a leading role in the delivery of high-quality patient care by setting standards of medical practice and promoting clinical excellence. The RCP provides physicians in over 30 medical specialties with education, training and support throughout their careers. As an independent charity representing 40,000 fellows and members worldwide, the RCP advises and works with government, patients, allied healthcare professionals and the public to improve health and healthcare.

Citation for this document

Royal College of Physicians. Caring for hospital patients with COVID-19: Quality of care in England examined by case record review. London: RCP, 2021.

Copyright

All rights reserved. No part of this publication may be reproduced, distributed, or transmitted in any form or by any means, including photocopying, recording or other electronic or mechanical methods, without the written permission of the copyright owner. Applications to reproduce any part of this publication should be addressed to the publisher.

Copyright © Royal College of Physicians 2021

ISBN 978-1-86016-851-2 eISBN 978-1-86016-852-9

Royal College of Physicians 11 St Andrews Place Regent’s Park London NW1 4LE

www.rcp.ac.uk

Registered charity no 210508



Contents Report at a glance ................................................................................................................................... 4

Foreword ................................................................................................................................................. 5

1. Executive summary ......................................................................................................................... 6

2. Introduction .................................................................................................................................... 9

3. Methods ........................................................................................................................................ 10

4. Demographic information from participating centres .................................................................. 14

5. Quantitative analysis, qualitative thematic analysis and vignettes .............................................. 18

6. Summary and conclusions ............................................................................................................ 37

7. Acknowledgements ....................................................................................................................... 40

8. References .................................................................................................................................... 41

The quality of care was scored based on case record review

The case records of patients with COVID-19 were reviewed

Report at a glance

Hospitals in

19 NHS trusts in England submitted data.This represents almost 19% of the English population.

These hospitals cared for 26,000 patients with COVID-19, 6,000 of whom died.

It was adequate, good or excellent for 96.5% of patients

96.5%

It was good or excellent for 77.4% of patients

77.4%

It was poor for 3.5% of patients

Whether patients did or did not receive critical care, and whether they survived or died, made no difference to the quality of care.

3.5%

Participating organisations

64% were men36% were women

Aged 22 to 100

510 patients



Foreword Hospital care of patients with COVID-19 during the early waves of the pandemic has come under the media spotlight in the past year with articles predominantly focusing on overwhelmed NHS staff, crowded wards, elective care backlog and families unable to visit their loved ones.

This study into the care of hospital patients with COVID-19 provides a welcome insight into the high quality of care experienced by the majority of people. The report uses data from patient records provided by reviewers from a large and representative sample of NHS trusts in England. Reviewers used a validated method based on the Structured Judgement Review methodology used successfully for retrospective mortality review.

One of the report’s key findings is the very few instances of poor care. This is exceptional given the enormous pressures that were very rapidly placed upon the NHS at the start of the pandemic. It is a cause for celebration of the efforts and expertise of NHS staff.

In this report, a series of patient vignettes bring to life the typical range of care experienced by many patients in hospital with COVID-19. These are compelling to read and provide key learning points to influence future care delivery. The study also introduces novel analysis of narrative using natural language processing in healthcare and the use of sentiment analysis to detect the emotional content of narrative – there is much to learn from this to support the qualitative analysis of case record reviews.

The study was made possible by a grant awarded from the RCP COVID-19 Appeal. I would like to extend sincere thanks to the many reviewers in NHS trusts and organisations who agreed to take part and took the time to assess patient cases. These efforts have provided invaluable data to form the basis for this study and its findings. This report is the result of collaboration with staff at Imperial College and the Association of Professional Healthcare Analysts who contributed significantly to the analysis of the data and emerging themes – thank you to them for this support. Finally, particular thanks must go to Dr Andrew Gibson for steering the report to publication during what continues to be a very challenging time.

The study underlines that learning from good and excellent care is as important as learning from poor or negative experiences. We hope it highlights the overall good quality of care experienced by COVID-19 patients in hospital during the pandemic and contributes to improving care for patients in the future.

Dr Sarah Clarke Clinical vice president, Royal College of Physicians



Executive summary This study used a validated, standardised methodology to assess the quality of care delivery of patients treated for SARS-CoV-2 (COVID-19). Retrospective case record review was performed in a large sample of hospitals in England in 2020/21 to understand the quality of their care and identify areas for learning. This is the largest study of the quality of care for patients with COVID-19 in the UK to date, other studies having focused on demographics, outcomes or specific treatments. In addition, the study intended to determine if there were differences in the care received by patients who either survived or died with or without a critical care stay

Data were received from 19 NHS trusts which were widely distributed throughout the seven health regions in England. Participating hospitals encompassed a mixture of large teaching hospitals and smaller district general hospitals. The median catchment population was 500,000, which equates to 10.5 million people in total or almost 19% of the population of England. These hospitals cared for over 26,000 patients with COVID-19 in 2020, the majority being admitted in the first wave of the pandemic between early March and early July 2020.

The study was designed to gather quantitative and qualitative information about the care received by these patients. A total of 510 patient case records were reviewed.

Sixteen centres returned data that allowed the detailed care scores to be calculated throughout all phases of care. Only 18 of 510 cases (3.5%) had an overall care score of poor. None of the cases where care was scored poorly affected the outcomes for the patients. Given the nature and scale of the pressures the pandemic very rapidly placed on the NHS, this is a testament to the efforts and expertise of staff.

A picture of the typical care received by the patients in this study is described in Vignette 1. It includes many of the recurring positive aspects of care experienced by patients during the COVID-19 pandemic in 2020.

Vignette 1

Typical ward care received A 79-year-old woman was admitted to hospital at 9pm following a fall and increasing shortness of breath. She was seen rapidly by a consultant within 2 hours of admission and COVID-19 was considered to be the diagnosis.

A rapid decision was reached that she should not be escalated to the ITU. This was well-documented and with the full agreement of the multidisciplinary team (MDT) and patient. All subsequent documentation was of a high standard. Contact with relatives was completed in a timely manner. While receiving ward care the patient was referred to the appropriate specialties with gastroenterology, diabetes and renal teams all involved. There was good awareness of the patient’s multiple conditions. Nursing care was of a high standard, as was associated documentation. Blood sugars were well-documented. Chest X-ray on admission showed mild bilateral patchy shadowing. The COVID-19 swab was positive, and the patient was treated on a COVID-19 ward with good infection prevention and control. There was senior involvement and MDT input at all stages and timely review from all specialties. She experienced a short period of hypoactive delirium while on the ward, which her relatives found distressing but was very well-managed by the team. She developed a low mood with some anxiety elements in the subsequent days and was mildly deconditioned which was minimised by high-quality physiotherapy input. She was discharged to the community for further recovery and rehabilitation prior to being transferred to her previous residence.



A subset of 216 patient case reviews supplied allowed more detailed comparison of the care quality scores across four scenarios. Whether the patient had received critical care or not, and whether they had survived or died. The patient characteristics of these four subgroups was also analysed.

From our detailed analysis we can conclude the following:

> This study presents the quality of care delivered from a large sample of patients with COVID-19 treated in representative NHS hospitals in England during the COVID-19 pandemic in 2020.

> The care delivered to patients with COVID-19 was of a high standard. It is depicted in a series of vignettes drawn from shared case studies and adapted to include common issues identified in the thematic analysis.

> Learning from the review of good and excellent care, which is much more prevalent, is as important as the learning which can be derived from poor care.

> There was no evidence for differential quality of care delivery between any of the four subgroups but those patients who died and were not escalated to critical care had poorer care scores across the phases of care. This may be due to a combination of hindsight bias and the absence of negative factors in the survivors who did not, for example, experience poor end-of-life care (EOLC).

> Care judged to be poor overall was uncommon. When it did occur, it was related to EOLC issues, nosocomial infection (the acquisition of COVID-19 in the hospital), delays in assessment, escalation and the two generic issues of poor communication and poor documentation.

> The experience of patients with a known learning disability appears to be different in respect of the rational for escalation decisions and a possibility that the quality of their EOLC experience was different. Due to low case numbers, however, this finding should be treated with some caution. There were clear examples where the reviewers were uncertain of the documentation and reasoning for important care decisions.

> In the context of adequate, good and excellent care many centres described a lack of consistency for important aspects of care delivery of fundamental healthcare including: — EOLC experiences — assessment in emergency departments — documentation and communication — senior review — DNACPR decisions — discharge planning

> A multidisciplinary approach and the importance of palliative care were uniformly celebrated and a key in the successful delivery of high-quality care to these patients.

> Care of patients with nosocomial infections were studied in 11 centres. These patients accounted for approximately 8% of the mortality reviews in this study. All but one of the centres described hospital-acquired infections as negative aspects of care. Many centres established investigations into the causes of these infections.

> There was a variation between centres of crude mortality due to COVID-19 from approximately one in 10 cases to approximately one in three cases. Further analysis is required to explain these differences.

In the few cases where the overall care was judged to be poor, there was clear evidence of action plans in the centres that recorded cases of nosocomial infection, usually via the Serious Incident Framework 2015.



Participating centres also identified areas of care which, although otherwise good or excellent, had room for improvement. The qualitative analysis was performed by manually reviewing (analogue sentiment analysis) and amalgamating the themes that each centre had described. Several of the centres’ reports are exemplars in this respect with robust analysis and strong actions described. This analysis identified 24 main themes, each shared by more than two of the centres. The manually extracted themes were reinforced by analysing the subset of 216 individual patient reports by applying natural language processing (NLP) algorithms using a combination of Latent Dirichlet Allocation (LDA) and computational sentiment analysis. Sentiment analysis created positive commentary on care but was unable to differentiate poor commentary from simple descriptions of symptoms.

The common themes occurring from these analyses have been incorporated into several case vignettes. The vignettes convey key findings about care delivery and are compositions created, with permission, from several similar case studies supplied from the centres. The majority of cases were in patients who did not survive their illness which influenced the themes created, but the analysis performed described issues which were applicable to all the patients in the study. The vignettes therefore reflect the outcomes described in the study for all patient groups.

Recommendations 1 Hospitals and teams that have delivered excellent care during the pandemic should analyse the

factors that have enabled this to happen. They should be encouraged to share the learning from the key findings, locally or nationally, to support more consistent quality of care in the future and to raise awareness across the NHS and with the general public.

2 Healthcare professionals and organisations should strive to reduce the variation in care in key areas including:

− end-of-life care in hospital − early assessment in acute care presentations − documentation and communication − senior review and decision making − treatment escalation planning, conversations and documentation − discharge planning − particular focus should be given to reducing variation for those with learning disabilities in

these areas.

3 The guidance and standards published by the RCP covering many of these areas should be implemented and followed more consistently.

4 All NHS organisations are encouraged to pay particular attention to the quality of care, decision making, communication and documentation that is required for vulnerable people, including those with learning disabilities.

5 The NHS should further explore the reasons for the range of hospital mortality between areas during the COVID-19 pandemic.

6 The RCP and Imperial College should further explore the role of natural language processing in the qualitative analysis of structured case record reviews.



Introduction A SARS-CoV-2 (COVID-19) pandemic was declared by the World Health Organization on 11 March 2020. One year later 117 million people had been infected and approximately 2.6 million people had died from the disease.1 On the first anniversary of the declaration of the pandemic, over six and a half billion articles had been written about many aspects of the disease and its global effects.

The most common description of the pandemic is that it is ‘unprecedented’. The pandemic and the response to it have not just affected population health. Far-reaching consequences have also affected the financial, retail, travel, sport, hospitality and social care sectors. Indeed, across the globe the normal structures within societies have experienced enormous upheaval during this period with resolution of some of these aspects likely to take years, if not decades, to address.

The health and social care structures within the UK have been subject to extreme pace of change. Rapid structural and governance changes were required in reacting to the pandemic as well as a plethora of rapidly written and extensive guidance to support the services in extremis. This included the creation of Nightingale hospitals in strategic areas.

The national level of alert within the NHS was at a maximum (Level 4) for most of the first wave of the pandemic with a clear command and control operational structure.

These multiple factors imposed enormous pressure on the NHS and social care, affecting its workforce and its patients. This has continued in an unrelenting manner throughout the course of the outbreak.

However, the pace of change and the unprecedented nature of the pandemic has also prompted some commentators to question the consistency of care delivery. This included the general preparedness of the NHS for the outbreak, the level and provision of personal protective equipment (PPE), the concerns over clinical decision making and escalation of care in vulnerable groups. Heslop described the early information that was available from analysis of COVID-19 deaths in a population of 50 patients with a known learning disability.2 In addition, some reports in 2020 raised other concerns about do not attempt cardiopulmonary resuscitation (DNACPR) decisions in the general and the learning disability population.3

This study used a standardised but slightly modified Structured Judgement Methodology (mSJR) format and common data set collection to understand more completely the care delivered to patients with COVID-19 within NHS trusts during the pandemic, comment on the emerging themes and create clear conclusions and recommendations. The participants were asked to review sets of patient case notes using the SJR methodology and then construct a ‘report’ which identified and described the themes that they observed. The main outcomes of the study reflect the mostly qualitative analysis of the narrative judgements and the thematic analysis reported by the participating centres using a combination of manual analysis of the themes described and a natural language processing algorithm. Vignettes have been constructed to articulate themes.

The quantitative elements of the study are restricted to the demographic data returned by the centres, which describe the representative nature of the sites involved and the analysis of care scores across the whole sample and the associated subgroups. Where low sample sizes were seen it restricted the inferences that could be drawn.



Methods Forty centres across the UK expressed an initial interest in participating in the study with 19 centres from England eventually providing data.

The methodology involved a minor modification to the validated SJR methodology. All the centres used the SJR methodology in the construction of their reports and, in addition, collectively supplied 241 individual anonymised modified Structured Judgement Review (mSJR) reports, which were subject to further quantitative and qualitative analysis. The majority of the patients in this study did not survive the illness, as sites were predominantly using SJR for mortality reviews. This does not reflect the overall survival experience of hospitalisation of patients with COVID-19 in the pandemic.

3.1 Modifications to the Structured Judgement Review The Structured Judgement Review (SJR) methodology4 is widely used by trained reviewers in the NHS as a means of understanding the quality of care delivered in relation to the ‘learning from deaths’ statutory requirement of trusts. Its primary role is in undertaking retrospective mortality review following the extensive training programme delivered by the RCP and partners across the UK as part of the National Mortality Case Record Review (NMCRR) Programme between 2016–19.5 Using this standardised review method that was very familiar to many clinicians helped to minimise variation in the case note review process. The methodology was modified for this study (mSJR) to enable anonymised data to be gathered on patients who also survived COVID-19 infection and were discharged from acute care.

In addition, the mSJR form collected information on COVID-19 status and whether patients had a learning disability and/or mental health issue. The modified form is reproduced in Appendix 1, published in a separate document available to download.

3.2 Data acquired

The analysis was performed by the RCP with significant contributions from data analysts at the Association of Professional Healthcare Analysts as well as a data team in the Institute of Global Health Innovation at Imperial College London, working independently via Imperial Consultants. The participating centres were asked to return standardised data and self-analysis of the themes that had emerged from the cases that were reviewed.

This resulted in both quantitative outputs and qualitative thematic analysis. The qualitative analysis employed two complementary methods – a manual review of the themes created and returned by the centres, and an analysis of 216 individual mSJR forms using a natural language processing algorithm. This overall process is visualised in Fig 1.



Figure 1: Flow diagram overview of the study.

*This is a subset of the 425 patient cases – 216 cases were subject to analysis NMCRR = National Mortality Case Record Review; mSJR = modified Structured Judgement Review



3.3 Quantitative analysis Data was analysed to demonstrate:

> the demographics and characteristics of the 19 participating centres (see Box 2 and Table 1) > comparison of the differences in ‘COVID-19 experience’ between the centres > the care scores calculated by each reviewer on a scale of 1 to 5 for the phases of care (admission

care, ongoing care and end-of-life or discharge care) and overall care score from 425 patient cases with mSJR

> subgroup analysis of care scores for the four subgroup scenarios.

3.4 Qualitative methodology for manually generated themes Each participating centre performed a thematic analysis of their own patient cases, creating a list of summary statements, comments, learning points and actions for each organisation. These were grouped around the phases of care.

Statements from all the centres were then systematically reviewed, the central theme of the statement identified, and its origin noted. A further description of each statement was also noted, and it was assigned as a positive or negative trait. This identified 24 themes across the phases of care in at least two centres. Themes were ranked by the frequency of positive and negative comments.

Sentiment analysis is the computational science of knowing the emotional content of words within narrative text.6 The method described above is an analogue form of sentiment analysis.

Box 1 shows a worked example of this manual sentiment analysis with respect to the theme of do not attempt cardiopulmonary resuscitation (DNACPR) orders.

3.5 Qualitative thematic analysis using natural language processing

The subset of 216 mSJRs were analysed using natural language processing (NLP) algorithms at the Institute of Global Health Innovation at Imperial College London. Latent Dirichlet Allocation (LDA) was used to understand if there were statistically different themes present in each of the four subgroups. Sentiment analysis was used to identify positive or negative emotional content of narrative. Further detail on these methods is described in Appendix 4.

The NLP outcomes were used where appropriate to complement both the quantitative care score analysis and the qualitative manual sentiment analysis.

Box 1: ‘There was limited discussion about DNACPR on occasion’

> the theme is DNACPR

> the description of the theme is of a ‘limited discussion’

> this is noted as a negative comment about DNACPR from this participating centre (even though applied to good care)

> negative comments about the theme of DNACPR were summated across centres

> a similar process occurred for all positive comments about DNACPR.



3.6 Vignettes

A series of nine vignettes was constructed to articulate both the key aspects of the care delivered and the commonly recurring events that led to the themes identified.

Eleven centres gave consent to use multiple case studies to articulate relevant themes. To ensure that the experiences of patients were described as completely as possible vignettes were created by amalgamating multiple patient cases with similar characteristics and narrative from the centres. No patient identifiable information is included as each vignette is a composite.

The vignettes are based upon the case studies supplied and are reflective of the study sample. The analysis of the patients who survived and those who died reveals a commonality of these themes across all patient care experiences.

These vignettes are used to report the key aspects of care that were observed from the review of the patient record and describe patients who died or survived with or without critical care input.



Demographic information from participating centres

4.1 Participating centres

The participating centres and their geographical spread are shown in Box 2. There were seven large teaching hospitals with medical schools and 12 hospitals that delivered acute secondary and tertiary care.

4.2 Catchment population, COVID-19 rates and mortality

Table 1 shows the total catchment population of the 19 participating centres and the total number of patients with COVID-19 seen at those centres at specified times. The number of patients who survived COVID-19 and the number who died in each centre allow a crude mortality calculation. The table also shows the minimum, maximum and median values for centres for each parameter where relevant.

Table 1. Total catchment population, numbers of deaths and the proportions of survivors in participating centres (n=19).

Demographic Number (total) Minimum Maximum Median Catchment population

10.46 million 233,303 1,016,000 500,000

Total patients with COVID-19

26,326 449 3077 N/A

Total deaths from COVID-19

6,389 125 813 N/A

COVID-19 survivors

19,937 390 2475 N/A

Crude mortality 24% 12% 35% 24% *The crude mortality is the total number of non-survivors in a specific time frame divided by the total number of COVID-19 cases in that time frame per organisation.

The total population of England in 2019 was estimated by the Office for National Statistics as 56.23 million.7 The catchment population of the study of 10.46 million therefore equates to approximately 19% of the population of England.

Box 2: NHS regions, cities and towns represented

N East and Yorks: Barnsley, Leeds and Sheffield

North West: Bolton, Southport and Ormskirk

East of England: Peterborough, Stamford and Norwich

Midlands: Lincoln, Grantham and Boston

London: Central London, Croydon, Enfield and Camberwell

South East: Reading, Newbury, Epsom, Canterbury, Margate, Brighton and Bournemouth

South West: Bristol and North Bristol



The total number of hospital deaths from COVID-19 in the centres was a minimum of 6,389 which therefore accounts for approximately one in seven acute hospital deaths from COVID-19 in 2020.

There is a range of 12% to 35% crude mortality between the 19 centres, with a median of 24%.

Docherty et al, on behalf of the ISARIC4C investigators,8 demonstrated that in a cohort of 20,133 admissions to hospital that the mortality within 28 days of admission for patients with COVID-19 admitted to ward-based care in the first wave was 26%. The equivalent metric for patients treated in critical care was 32%. The crude mortality calculated for this study is consistent with these values.

The differences in crude mortality between centres will be multifactorial. These include population demographic differences but may also include differences in service delivery. Systematic decisions about conveyance to hospital and admission decisions within emergency departments may have been a factor. Fig 2 shows the lack of correlation between crude mortality and the proportion of the catchment population over 65.

Figure 2: Crude mortality against proportion of catchment population over 65 years.

There is no correlation between crude mortality and the size of the organisation or catchment population, or the proportion of residents from ethnic minority backgrounds in the catchment area of the hospital (Fig 3 and 2011 census data).

R² = 0.0471

0

5

10

15

20

25

30

35

40

0 5 10 15 20 25 30 35

Crud

e m

orta

lity

% Over 65 in population

Crude mortality against proportion of catchment population over 65



Figure 3: Crude mortality expressed against the proportion of population from ethnic minority backgrounds in each catchment area.

4.3 Patient characteristics

Table 2 shows the age and sex demographics of the patients included in the study. These demographics are also shown for those with learning disabilities.

Table 2: Age and sex of the study population (n=435), and those with learning disabilities (n=18).

Characteristic Men (%) Women (%)

Sex overall (n=435) 279 (64%) 156 (36%)

Age range 23–99 years 22–100 years

Range of medians per centre 72–85 years 72–85 years

Sex (learning disability) (n=18) 12 (67%) 6 (33%)

Median age (learning disability) (n=18) 60 years 33 years

Age range (learning disability) 42–73 years 22–67 years

Median length of stay (range in days) 9 (0–123) 7 (0–114) The ratio of men to women was approximately 2:1 with very similar age ranges for both sexes and median age ranges of 72–85 years. The median age of patients with learning disability was 47 years, which is 25 years lower than the whole sample. The five youngest women in the whole sample all had a known learning disability.

Age analysis of the four subgroups is shown in Fig 4.

R² = 0.0021

0

5

10

15

20

25

30

0 10 20 30 40 50 60 70

Crud

e m

orta

lity

% citizens from ethnic minorities in population

Crude mortality against proportion of population from ethnic minority backgrounds



Figure 4: Age analysis of the four main mSJR subgroups (n=216).



Quantitative analysis, qualitative thematic analysis and vignettes

The analysis is divided into different sections that come under overarching analytical processes, which can be described as:

1 Quantitative analysis of the care scores derived from the data returns

2 Qualitative analysis of themes created by both manual inspection of narrative and a natural language algorithm

3 Construction of vignettes.

5.1 Care scores

The care scores are linked to the explicitly judged care quality for phases of care and overall care. Care scores from 425 patient cases are shown in Table 3.

In the subset of 216 patient cases comprising the four subgroups (those who died with or without critical care, and those who survived with or without critical care), a further subgroup analysis was also possible for patients with a known learning disability and patients undergoing an operative procedure in the context of critical care delivery.

Table 3 shows the care scores for the four phases of care described in the whole dataset of 425 patients on whom detailed information was available.

Table 3: Phases of care scores for admission, ongoing, EOLC/discharge and overall care.

Phase of care and care score

1 (%) Very poor

2 (%) Poor

3 (%) Adequate

4 (%) Good

5 (%) Excellent

Admission care (n=360)

1 (0.3)

5 (1.4%)

61 (16.9%)

180 (50%)

113 (31.4%)

Ongoing care (n=354)

0 19 (5.4%)

61 (17.2%)

175 (49.4%)

99 (27.9%)

EOLC or discharge (n=361)

0 17 (4.7%)

55 (15.2%)

173 (47.9%)

116 (32.1%)

Overall care (n=425)*

0 18 (4.2%)

78 (18.4%)

213 (50.1%)

116 (27.3%)

* The number of patient cases in each phase of care varies as some patients died rapidly in the first 24 hours.

There were no patients for whom the overall care was judged to be very poor. Admission care was adequate or above in 98.3% of 360 cases, ongoing care was adequate or better in 94.6% of 354 cases, EOLC or discharge care was adequate or above in 95.3% of 361 cases and overall care was adequate or better in 95.8% of 425 cases.

It is therefore clear that the care given to patients with COVID-19 was overwhelmingly of an adequate level or above in this representative study using a mSJR case record review.



Vignette 1 describes the excellent care seen repeatedly in the cases reviewed from the participating centres and identified within the thematic analyses.

In the 425 patient cases individually identifiable out of the overall 510 cases, there were 18 patients judged to have received poor overall care.

A study by Hogan et al 9 showed that 3.5% of care did contribute to the death of patients, while Roberts et al, in describing a cohort of over 7,000 retrospective case record reviews using the PRISM 2 methodology, determined that less than 0.5% of care was felt to directly contribute to acute hospital deaths.10 The inference from these two studies is that the frequency of poor care in acute hospitals is between 0.5 and 3.5%. The mSJR does not identify if poor care contributed to the death of the patient.

The commonest causes of care being judged poor are described in Box 3. Approximately three-quarters of poor care was ascribed to documentation, communication and nosocomial infection and the remaining quarter was due to EOLC issues, delays in assessment or lack of evidence for escalation decisions.

Vignette 1

Typical ward care received A 79-year-old woman was admitted to hospital at 9pm following a fall and increasing shortness of breath. She was seen rapidly by a consultant within 2 hours of admission and COVID-19 was considered to be the diagnosis.

A rapid decision was reached that she should not be escalated to ITU. This was well-documented and with the full agreement of the multidisciplinary team (MDT) and patient. All subsequent documentation was of a high standard. Contact with relatives was completed in a timely manner. While receiving ward care the patient was referred to the appropriate specialties with gastroenterology, diabetes and renal teams all involved. There was good awareness of the patient’s multiple conditions. Nursing care was of a high standard, as was associated documentation. Blood sugars were well-documented. Chest X-ray on admission showed mild bilateral patchy shadowing. The COVID-19 swab was positive and the patient was treated on a COVID-19 ward with good infection prevention and control. There was senior involvement and MDT input at all stages and timely review from all specialties. She experienced a short period of hypoactive delirium while on the ward, which her relatives found distressing but was very well-managed by the team. She developed a low mood with some anxiety elements in the subsequent days and was mildly deconditioned which was minimised by high-quality physiotherapy input. She was discharged to the community for further recovery and rehabilitation prior to being transferred to her previous residence.

Box 3: Commonest causes of an overall care score of poor (n=18)

> Poor documentation

> Poor communication

> Nosocomial infection and hospital-acquired COVID-19

> End-of-life care issues

> Delayed assessment

> Escalation decisions



5.2 Manual analysis of themes

The methodology for manual analysis of the themes is described fully in section 4. It is important to recognise that the negative commentary is equivalent to ‘what could have been improved’ but this is not synonymous with ‘poor care’. The manual analysis revealed 24 themes, each of which were reported by at least two different centres. Table 4 lists these themes together with the frequency of positive or negative comments made by centres.

Table 4: Themes, with number of centres giving positive and negative comments.

Theme (abbreviated)

Positive comments from centres

Negative comments from centres

EOLC 18 15 Escalation decisions 15 14 Assessment/admission 14 12 Communication 14 10 Senior review 12 6 DNACPR 10 5 MDT working 10 1 Palliative care 11 2 Use of pre-emptive prescribing 7 2 Discharge planning 4 4 National guidance 6 0 Documentation 7 4 Family issues 4 1 Challenging conversations 3 3 Compassion 3 0 MCCD issues 2 2 Visiting 1 7 Treatment 1 2 Death quality (not EOLC) 1 4 Infection control 0 3 Transfer 0 8 Mental capacity/DoLS 1 1 Fluid balance 0 5 Hospital-acquired COVID-19 0 11

MCCD = medical certificate of cause of death; DoLS = Deprivation of Liberty Safeguards

A number of themes attracted positive and negative comments in almost equal measure, implying significant variability of care quality related to that theme. These are areas where quality improvement activities are warranted to reduce this variation.

The themes are in part generated by the study methodology which specifically encouraged participating centres to concentrate on decision making and communication. The mSJR also has a specific section on EOLC and it is therefore not surprising that this features in the thematic analysis. As previously noted, the common themes identified in poor overall care scores (Box 3) are identical to the commonest themes created from the analysis of the whole sample, which also considered adequate, good and excellent care.

Themes were aggregated into four broad groups and are shown in Table 5.



Table 5: Themes categorised into four main groups.

Common themes with significant variation

EOLC experiences Escalation decisions Assessment and admission care Communication and documentation Discharge planning DNACPR decisions Senior reviews Challenging conversations

Themes attracting mostly positive commentary

MDT planning Palliative care Compassion National guidance Family issues Pre-emptive medication

Themes attracting mostly negative commentary

Nosocomial infection Transfers of patients Visiting Fluid balance Quality of death (not EOLC)

Themes from <20% centres Infection control Treatment (options) Mental capacity and DoLS MCCD issues

DoLS = Deprivation of Liberty Safeguards; MCCD = medical certificate of cause of death

The themes exhibiting the greatest inconsistency include the most fundamental aspects of healthcare such as documentation and communication. Other key themes have been the subject of RCP reports and recommendations; these include assessment, decision making, DNACPR decisions, timing of senior review and EOLC.

The role of palliative care, MDT input and the use of pre-emptive medication during the pandemic were reported very positively by the centres with numerous examples of rapid recognition of the

Vignette 2

Poor communication A 57-year-old man was admitted to the emergency department from a hostel via his GP. Initial and ongoing care was of a high standard, but the patient deteriorated and died rapidly. He had no next of kin with him at death.

Communication with the patient’s family was poor during the whole period – there was no evidence from review of the medical records of any contact with them, despite there being a contact number in the medical record of a sister. The hostel manager contacted the family to inform them of the death and they then telephoned the ward with questions. This may have resulted in additional distress to the family. While the care of the patient was of a high standard there was clear room for improvement in communication with his next of kin.



end of life. Almost 90% of centres commented that EOLC was of a very high standard. On occasion, and sometimes in the early phases of the pandemic when the speed of deterioration wasn’t as well understood, there was delay in recognition that a patient was dying. When this occurred, the reviewers tended to score EOLC poorly and, as already seen, EOLC featured in those cases judged to be poor overall (Box 3). However, identifying those aspects of EOLC and palliation that could have been better was also very prominent in the reviews, with almost 80% of centres recognising a need for improvement in this aspect. One report captured eloquently the key issue: EOLC, when not done well, had a disproportionately large impact on both the patient and the family. Vignettes 3 and 4 demonstrate this variation.

DNACPR was raised by 79% of the centres with the majority commenting in a positive manner about several aspects of DNACPR orders, which were often well-documented and sensitively approached. However, 26% identified some concerns about DNACPR, mainly in the context of documentation and rationale for the decision reached. There was no clear evidence of differences in the application of such decisions across the subgroups, although most of the negative commentary about DNACPR was within the subgroup of patients who died without escalation to critical care (see later). This is captured within the vignettes.

Vignette 3

Compassion in EOLC delivery An 82-year-old man was admitted via the ED with shortness of breath, chest pain and cough. He was seen rapidly by the respiratory consultant. The patient had COPD with limited exercise tolerance. A diagnosis of likely COVID-19 with possible superadded bacterial infection, and probable element of heart failure was made. Following well-documented discussions with the patient, it was agreed that the patient was probably in the last days of life and that he would not be a candidate for invasive ventilation. There was rapid input from palliative care and the patient was given pre-emptive medication as needed.

Due to the high incidence of COVID-19 his family did not wish to visit. However, the F2 doctor and nursing staff took turns to sit with the patient and read messages that the family had sent. It should be recognised that during the height of the pandemic staff went above and beyond to help comfort this patient in lieu of family.

Vignette 4

Senior reviews and delays in EOLC recognition An 88-year-old man was admitted following a collapse at home. He tested positive on admission and was transferred to a community hospital bed after discussion.

Due to the patient’s mental health, he was seen frequently by the mental health liaison team during his stay. However, reviewing the patient’s notes it is apparent that he was only seen three times by a medical consultant during the last month of his life. During the height of the pandemic it is noted that consultants did not attend the community hospital with the same frequency, which in the opinion of the reviewer undoubtedly had a negative impact on the patient’s care. Communication with the family could have been better during this period, and end-of-life care would probably have been initiated sooner had there been more input from medical consultants.



Analysis also demonstrates clear evidence that most patients in the pandemic were rapidly assessed. The care scores from the whole study reflect this with only 1.7% of care judged to be poor or very poor in the admission phase (Table 3). However, over 25% of the centres did also consider that on occasion there were clear delays in this process, although importantly, none of the reported delays were felt to have affected the outcomes for the patients. Despite the nature of the pandemic and the lack of experience with an unknown pathogen, assessments were consistently prompt, thorough and of high quality.

Several participating centres described issues with the rapid reorganisation of their hospitals due to COVID-19, which interfered with the usual operations. This included repurposing of wards as well as redeployment of staff to unfamiliar duties, which often included consultants. Examples where this restructuring had unintended consequences for the delivery of healthcare allowed reviewers to explicitly comment on this issue and are captured in vignettes.

Vignette 5

Prompt senior reviews and rapid escalation decisions A 77-year-old man was admitted from a local care home with increasing shortness of breath. He had been refusing food at his care home for 3 days. He had a rapid assessment in the ED and was immediately suspected of having COVID-19. His admission swab proved positive for the virus. There was very early senior consultant review but unfortunately his condition deteriorated rapidly and his oxygen requirement increased. In view of his multiple conditions there was a well-described and sensitive discussion about escalation, and it was agreed that ward-level care was appropriate. There was clear evidence of a discussion about a DNACPR decision taken in conjunction with the consultant and family. The poor prognosis was conveyed well and the decision to involve palliative care staff early was made.

The palliative care team rapidly assessed the patient and pre-emptive medication was prescribed and given as needed. There was evidence of daily communication with the family and sadly the patient died on day 8 of his admission with his family and chaplain present.



All but one of the 63% of centres that described patients with nosocomial infection reported care negatively. Additionally, 42% of centres cited concerns about the inappropriate transfer of patients being relevant to the nosocomial infections. There were 43 patients described as having hospital-acquired COVID-19 and 19 mSJRs were provided which enabled a more detailed understanding of the care delivered to these patients. The majority were in mortality reviews, and 16% of patients in the study with hospital-acquired COVID-19 were in the survivor group. This implies that, in this sample, one in 12 patients who did not survive the illness had acquired it as an inpatient. These data cannot be extrapolated to estimate the mortality from nosocomial infection nationally.

Nosocomial infection has become an issue of national concern and inquiry, with current estimates as high as 25% in UK healthcare organisations.11 It is likely that there is a wide range of nosocomial infection rates with some UK centres having rates as high as 33%.12 The incidence of nosocomial infections will require significant attention in any future COVID-19 planning.

There was very clear evidence in the reports supplied for the study of robust action plans to address the issue of nosocomial infection, usually via the Serious Investigation Framework (2015).

Vignette 6

Delays, assessment, senior review and communication An older man aged 86 was admitted via the ED and waited around 8 hours to be assessed. He had mild cognitive impairment and lived alone. He had four carers a day visiting and a son living close by.

He was eventually seen on the post-take ward round (PTWR) but a chest X-ray wasn’t carried out until after the PTWR. The reviewers felt the X-ray should have been done before the patient went to the ward due to the potential disturbance to an older patient with cognitive impairment and the obvious infection risk. The reviewers did not feel there was sufficient information documented on the PTWR.

Throughout the patient’s care junior staff reported being uncertain about which staff to approach for patient management decisions. This was thought to be a consequence of the major reorganisation of hospital staff introduced to cope with the expected surge in COVID-19 cases.

There is one entry outlining a discussion with the family about the diagnosis, which was undertaken by an internal medicine trainee. The patient had a sudden deterioration but there are insufficient notes about it. Although the medical care was managed appropriately, there was a lack of documentation relating to the consultant or senior involvement.

From reviewing the patient’s notes, it is not clear how much senior input there was over the course of 2 days. There was no clear nursing care plan. Overall, communication with the patient and his family could have been better.



A theme that emerged from 42% of centres related to visiting arrangements during the pandemic. Issues were raised by patients, family members and carers about access to both the wards and intensive care units. It is clear from the reports returned that organisations developed safe and compassionate ways to maximise the contact between patients and their relatives and carers using a combination of methods to connect all parties. On occasion this led to very challenging conversations which were clearly distressing for both the relatives and staff who were simultaneously attempting to maximise communication and restrict footfall in line with the national guidance issued and infection risk.

A number of important issues were identified that were relatively uncommon and on occasion unique to a centre. These included concerns about the variable application of the Mental Capacity Act, family concerns about COVID-19 appearing on the medical certificate of cause of death (MCCD), and the absence of meaningful treatment options in the very early days of the pandemic.

Several individual mSJR reviews articulated the notion that some patients would have survived the illness in the early weeks had steroid therapy and anticoagulation been available earlier. This would be true of any retrospective analysis on available treatment options in healthcare. However, the study also established that the majority of individual reviews described the strict adherence to national guidance, although this was mentioned as a theme by only a third of centres. Some concerns were expressed about appropriate fluid balance and 25% of centres described this as a theme, especially in the early weeks of the pandemic. This was particularly noticeable in frail older patients.

Although mentioned by only one centre as a theme, inspection of the case reviews of the 216 mSJRs identified that a significant proportion of escalation decisions were discussed in terms of the ‘ceiling of care’. This was usually meant to convey the notion of ward-based care versus the escalation of care to a critical care unit. Many centres referred to ‘treatment escalation plans’ (TEPs) while a minority used the ReSPECT nomenclature for the same decision-making process.

Vignette 7

Multiple transfers and hospital-acquired COVID-19 A 63-year-old male patient was transferred from the regional cardiac centre having had an elective admission for aortic valve replacement. He had a negative COVID-19 test at the tertiary centre and was re-swabbed on day one and day five of his admission, and again these were both negative. He required a few days of further input as his diabetes had become unstable postoperatively with a background history of atrial fibrillation and hypertension. He had five ward moves because of significant operational pressures within the trust.

There was a delay of 10 days in organising a package of care and on day 9 he developed a temperature, cough and shortness of breath with arterial blood gases suggesting significant hypoxia. He was re-swabbed and found to have COVID-19 and immediately transferred to the COVID-19 ward and reviewed by ITU staff. He was then transferred to intensive care and required ventilating rapidly after the transfer.

His family were allowed to visit in pairs as his condition deteriorated rapidly. Despite maximal organ support he died on day 14. The reviewers felt that this was poor care as he had clearly contracted COVID-19 in the hospital after multiple ward moves. A serious incident was declared which was fully investigated.



We recommend that that a nationally agreed nomenclature be adopted in describing the process whereby a patient’s care is subject to the discussion of escalation. This would be helpful to practitioners and the public.

5.3 Subgroup analysis

The subgroup analysis of the care scores was carried out on the following:

1 Patients who did not receive critical care and died 2 Patients who received critical care and died 3 Patients who did not receive critical care and survived 4 Patients who received critical care and survived 5 Patients with a learning disability 6 Patients undergoing a procedure

5.3.1 The analysis of the four main subgroups

As previously discussed, a total of 241 individual mSJR forms were returned by centres which allowed for a subgroup analysis of four specific scenarios in 216 of these cases. These scenarios are described in Fig 1.

Table 6 describes the care scores within the whole subgroup of 216 mSJR cases and Table 7 shows the scores within the individual specific subgroups.

Table 6: Overall care scores from the 216 mSJR cases.

disc = discharge care



Table 7: Care scores from the four main subgroups.

disc = discharge care

The proportions and distribution of the care scores in each phase in the subgroups (Table 7) are very similar to the whole data set described in Table 6 because they are extracted from the larger sample.

As can be seen in Table 7 the quality of admission care across the four subgroups was of an adequate, good or excellent standard in 99% of the patients admitted, with only two cases not in this category. This does need to be put in context. The great majority of the cases in this study were admitted during the first wave of the pandemic which saw unprecedented pressures on the NHS in general and on emergency departments in particular. This suggests that despite these pressures, emergency departments were still able to provide high-quality care to patients with COVID-19.

In contrast to the admission phase of care reviewers had concerns about the EOLC in those patients who died without admission to critical care, and this was significantly different to the judgements of EOLC in a critical care setting. However, the proportion of the excellent care received in this group was over 30% which also shows that almost a third of patients received the highest quality of EOLC.



This supports the hypothesis that there is a clear inconsistency or variability in the delivery of EOLC in the study population.

This subgroup analysis demonstrates that there was no care judged to be poor or very poor in the 70 patients who were admitted to critical care and either survived (n=21) or died (n=49). The provision of critical care was always of an adequate standard or above and for the most part the overall care was deemed good or excellent with only one case out of 70 scored as adequate (1.5%).

Reviewers judged that none of the survivors had poor overall care scores regardless of whether they received critical or ward-based care.

The apparent difference in overall care scores in the subset of 216 patient cases for those who died and received only ward-based care (n=114) compared with those who died and received critical care and all those who survived (total n=102) deserves comment.

There was no overall poor care in the latter two groups and only 4.4% (five patients) of the 114 patients who died having received ward-based care only were judged as poor. The poor care was judged to have been due to either EOLC experience, nosocomial infection or lack of evidence in making an escalation decision. In the larger cohort as previously described, 45% of cases where care was judged as poor were attributed to either poor EOLC experience or the fact that patients acquired COVID-19 in hospital.

It is possible that these factors were supplemented by hindsight bias in the judgements. Hindsight bias is a well-established phenomenon in which the outcome of a judgement is dependent on the nature of the outcome,13,14 ie looking at care of those who survived more positively than those who died.

Vignette 8

Critical care experience A 65-year-old Asian man with hypertension and diabetes attended the emergency department in extremis with severe hypoxia. Prior to his admission his health had been good and he was completely independent. He was seen rapidly by the ITU team within 35 minutes of arrival and an escalation plan was immediately created. He was intubated in the resuscitation area and quickly transferred to ITU. His chest X-ray was indicative of COVID-19.

NICE critical care guidance was strictly adhered to. His family were very anxious on attending at the height of the first wave of the pandemic and staff used iPads to have facetime with his family on the days they could not visit to explain what was happening to him.

He had a prolonged stay on critical care, during which he developed repeated pulmonary emboli, but he was eventually discharged to the ward on day 35. He was significantly deconditioned and required intensive physiotherapy input followed later by occupational therapy input. He needed a complex package of care to return home and his total stay in hospital was 75 days.

Further follow up established that he had found the ITU experience difficult and was referred to the multidisciplinary long COVID-19 clinical service for psychological input.



5.3.2 Patients with a learning disability

The characteristics of 18 patients in the sample who had a known learning disability is described in Table 8 and the care scores of that group of patients are described in Table 8.

Table 8: The demographic description of patients with a known learning disability.

Characteristic Value Male:female ratio 12:6 Median age all (range) 47 (23–73) Median age (women) 33 (22–67) Median age (men) 65 (23–73) Proportion who survived 17.5%

The male to female ratio of this subgroup reflects that of the whole group at 2:1 men to women, but as already described there is a significant difference in presentation age profiles of this group.

Vignette Vignette 9

Documenting decisions, delays and escalation A 46-year-old woman with a known learning disability was admitted acutely from her sheltered and warden-controlled accommodation where she was fully independent. Her BMI was 34. She was seen in the emergency department (ED) with shortness of breath and it was assumed she had COVID-19. She was not reviewed by the admitting team for 12 hours, but an X-ray carried out by the ED team showed typical features of COVID-19 pneumonia.

There is evidence of a discussion about DNACPR on the post-take ward round, but it is not clear and not documented on what basis the decision to restrict to ward-level care was made.

This decision was reviewed by ITU on the next day as her oxygen requirement had markedly changed overnight. The ITU consultant reversed the decision, and she was immediately transferred to ITU for airway management and immediately nursed prone which she found very difficult to tolerate.

Communication via Microsoft Teams was made available to her family. However, concerns were expressed by the family about this and they attended in person after a clearly challenging conversation which was well-documented.

She required intubation on day 3 but her condition deteriorated over the next 5 days despite high-quality medical care.

The family were concerned about the initial delay in being seen and the matron arranged a meeting with the medical team, which was well-documented. Sadly, she did not respond to treatment and died in the ITU on day 9 with two family members present.



Table 9: The care scores of patients expressed as a percentage of the group with a known learning disability in bold compared with the whole study sample in parentheses (n=18 and n=425*).

Phase of care/ care score

1 Very poor

2 Poor

3 Adequate

4 Good

5 Excellent

Admission care 0% 6% (1.4%)

6% (16.9%)

76% (50%)

12% (31.4%)

Ongoing care

0% 6% (5.4%)

23.5% (17.3%)

53% (49.4%)

12% (27.9%)

EOLC

0% 17.5% (4.7%)

17.5% (15.2%)

53% (47.9%)

12% (32.1%)

Overall care

0% 6% (4.3%)

23.5% (18.3%)

59% (50.1%)

12% (27.3%)

*See also Table 3 for comparison

There were 18 patients in the whole sample of 510 patients who had a known learning disability. The prevalence of learning disability in this sample is therefore 3.5%. It cannot be extrapolated whether this is representative of the wider pandemic and hospitalisation in England.

The proportion of annual deaths of patients with a learning disability prior to COVID-19 was approximately 132 per 10,000 or 1.32% per annum.15 Both the Learning Disabilities Mortality Review (LeDeR) programme and the COVID-19 patient reporting system (CPNS) have suggested that during COVID-19 there has been significant under-reporting of deaths in patients with a learning disability 16 with crude mortality estimated as over six times that of the general population.17

The median age of death of this group of patients was 47 years, which is 25 years lower that the median age of the whole sample. The median age in women with a known learning disability at death was 33, which is 30 years below the pre-COVID-19 median age of death for women with a known learning disability.2 The median age of death for men with learning disability is 65 years, which is the same as that expected before the pandemic.2 The small sample size means that any explanation of this difference should be treated with caution.

Analysis and comparison of the care scores described in Table 9 implies that the overall care was comparable to the whole sample analysed. Fifteen of the patients with a known learning disability died and four female and one male patient were ventilated. One patient had acquired COVID-19 in hospital. One patient had their overall care judged as poor.

However, the EOLC experience in this group of patients may have been different to that of the whole population with a threefold increase in poor care scores in the EOLC phase compared with the rest of the patient sample. This is based on only three patient cases and should be interpreted with caution.

5.3.3 Patients undergoing a procedure

The care delivered to patients who underwent a procedure was confined to those who received critical care. Table 10 describes the care scores of this group of patients.

Table 10: Care scores of those patients who underwent a procedure (n=42).

Phase of care/care score

1 Very poor

2 Poor

3 Adequate

4 Good

5 Excellent

Procedure 0 0 4 20 18



As shown in Table 10, all 42 patients who underwent a procedure received care judged to be adequate or better. Much of the procedural care was delivered in the setting of high dependency and critical care. Not all patients admitted to critical care were deemed to have had a procedure. Among the 42 patients, there were three comments that related to difficulties in managing proned patients with respect to skin care and intubation tube displacement.

5.4 Thematic analysis using natural language processing

The subset of 216 mSJR individual patient cases extracted from the whole sample were subject to analysis using a natural language processing (NLP) algorithm performed at the Institute of Global Health Innovation at Imperial College London as described in the methods section and Appendix 4 (see separate document).

Where appropriate, the NLP analysis supplemented and was complementary to the quantitative and qualitative outcomes described in sections 5.3 and 5.4 and, where relevant, added to the evidence to substantiate the previous conclusions drawn.

Using the results obtained from the Latent Dirichlet Allocation (LDA) analysis of the admission phase of care as an example (Fig 5), the first theme is found to include words associated with ‘admission’ and so this theme has been assigned the label ‘admission phase’. This process was repeated with each of the separate themes created in the phases of care to derive clinically meaningful labels which described all the themes across all five phases of care.



Figure 5: The eight themes defined in the admission phase of care.

0 0.01 0.02 0.03 0.04

positiveconfuse

symptomprevious_admission

liveescalation_plan

previoushospitaltransfer

admission

Admission

0 0.005 0.01 0.015 0.02 0.025

evidencetreatment

dnacprclerknote

completeassessment

patientplan

ed

Initial assessment

0 0.01 0.02 0.03

hourcritical_care

criticalquickly

itufamilyarrival

teamreview

medicalconsultant

care

Initial management

0 0.005 0.01 0.015

treatacute

includeassessment

contactconsultant_review

documentitu

documentationreview

discussion

Documentation

0 0.005 0.01 0.015 0.02 0.025

swabbreathunwellcough

formhome

historycovid

patientadmit

COVID-19 admission recognition

0 0.005 0.01 0.015 0.02

falldaughter

ward_roundsenior

post_ward_roundpost_ward

rounddischarge

postinitialward

Early ward care

0 0.005 0.01 0.015 0.02 0.025

sepsisceilcxr

identifyescalation

edcare

excellenthour

covid

COVID-19 decision to admit

0 0.002 0.004 0.006 0.008

riskpneumonia

dementiahistory

nurseblood

ivdiscuss

antibioticfluid

Patient treatment



Table 11 describes the themes from the five phases of care analysed in the 216 mSJR cases using the LDA modelling of the data.

Table 11: Phases of care themes derived from LDA analysis

Phase of care Phase of care themes Admission 1. Admission phase

2. Initial assessment 3. Initial management 4. Documentation 5. COVID-19 admission recognition 6. Early ward care 7. COVID-19 decision to admit 8. Patient treatment

Ongoing 1. Patient centred 2. Family engagement 3. Review 4. Ongoing treatment 5. Ongoing management 6. Deterioration 7. Ward discharge processes

EOLC 1. Treatment escalation planning 2. Family engagement 3. Palliative care 4. EOLC quality 5. EOLC decision making 6. EOLC management 7. EOLC treatment 8. Patient death

Discharge 1. Discharge documentation 2. Discharge plans 3. Follow-up plans 4. Family involvement and engagement 5. Care overview 6. Discharge advice

Overall 1. Medical treatment 2. Decision making 3. Care delivery 4. Care management 5. Family engagement 6. Documentation 7. COVID-19 status

The analysis then allowed for the statistical comparisons of each of the themes with respect to the variables of survival status and critical care status.

Statistically significant differences between the themes and the survival and critical care status variables were observed in all of the phases of care except discharge of survivors and this information is presented in Table 12. The detailed analyses appear in Appendix 3 (see separate document).



Table 12: An overview of themes that were more likely to be represented within phases of care from comparison of patients who died or survived with or without critical care escalation.

Phase of care Died Survived

Patient admission Initial assessment, documentation COVID-19 admission recognition

Ongoing care Family engagement, patient-centred care Ward discharge process, deterioration

Overall assessment Care management, family engagement

EOLC Not applicable Not applicable Discharge from hospital Not applicable Not applicable

Received CC Did not receive CC Received CC Did not receive CC

Patient admission Initial assessment

Ongoing care Deterioration Patient centred care Deterioration Patient centred

care, ward discharge process

Overall assessment Patient care

EOLC EOLC treatment

Palliative care, EOLC management Not applicable Not applicable

Discharge from hospital Not applicable Not applicable.

CC = critical care; EOLC = end-of-life care

This complementary analysis and overview suggests that COVID-19 admission recognition themes seem to be more prevalent in those who survived than in those who died and that initial assessment themes seem to be more prevalent in those who died. Clinically, this suggests that the early assessment in deciding which patients were escalated to critical care versus ward-based care was a key event. The manual analysis confirms that this was done very well in the majority of cases. Where there was commentary about areas for improvement, delay in the admission decision making process was often cited as the cause. Indeed, the analysis showed that delay in assessment was a component when the overall care was judged as poor. The care scores for the admission phase of care also support this aspect as crucial with only 1.7% of all the cases attracting a care score of less than adequate.

Family engagement appears earlier for those who died, usually within the early ongoing phase of care. This suggests appropriate patient-centred discussions were had between clinical teams and families when the prognosis was felt to be guarded or poor. Family engagement was prominent in the survivor cohorts later and towards the discharge phase of care, and discharge process themes are more associated with patients who survived. This is not surprising in the cohort of those who survived.

Deterioration themes were associated with critical care irrespective of survival or death and this reflects the decisions often made rapidly when patients deteriorated despite maximal organ support.

A further set of analyses was performed to see whether there were significant differences detectable in the phases of care in patients judged to have received ‘good care’ (care scores 3–5) versus ‘poor care’ (care scores 1–2). In addition, the cases of patients with a known learning



disability and those with nosocomial infections were also subject to this analysis. These three sub-analyses appear in Table 13.

Table 13: Analysis of phases of care against the variable of poor and good care, learning disability and nosocomial infection.

Patient care score Learning disability Nosocomial infection

Phase of care

Good care Poor care Learning disability

No learning disability

Nosocomial infection

No nosocomial infection

Patient admission

COVID-19 admission

recognition Admission

phase Initial

management

Ongoing care Review,

deterioration Ongoing

treatment

Ward discharge process

Overall assessment

Family engagement Documentation Documentation

EOLC Patient death Family engagement

Discharge from hospital

The analysis of the data in Table 13 especially with respect to a known learning disability (one case) and acquisition of COVID-19 from nosocomial infection should be interpreted with caution given the low numbers of patients involved in each group. This caution was also exercised in the analysis of care scores.

Good care appears to have had a higher likelihood of including themes related to COVID-19 admission recognition, care review, deterioration and family engagement. In contrast, the patients who received poor care had a higher likelihood of including themes related to ongoing treatment, documentation and patient death. This analysis supports the observations made in both the quantitative care score analysis and the qualitative manual ‘sentiment’ analysis where it was established that documentation and EOLC were judged to contribute to poor overall care outcomes.

This analysis suggests that patients with a known learning disability had a higher likelihood of including documentation themes. It is difficult to ascribe clinical meaning based on this analysis alone, but it does suggest that documentation in patients with a known learning disability was significantly different to the non-learning disability group. However, this could support the observations in other sections of this report which suggest that documentation of decisions was sometimes unclear in patients with a known learning disability.

Reports of patients with nosocomial infections had a higher likelihood of including themes around admission phase and ward discharge process, which is supported by the care score data and manual thematic analysis.



5.6 Natural language processing and sentiment analysis

The sentiment analysis performed within each phase allowed for the extraction of positive and negative phrases. Examples of the phrases obtained using this technique showed that many positive statements were made. Unfortunately, the methodology employed meant that some phrases from the narrative in each phase where symptom description occurred were perceived by the algorithm as negative comment. Manual inspection of these phrases confirmed, in many cases, that the sentiment was not truly negative and simply an account of patient symptoms. This is a potential limiting factor in the utility of sentiment analysis in interrogating healthcare narrative.

The example of sentiment analysis for the admission phase of care described in Table 14 articulates this methodological issue.

Further detail of the results of this analysis is given in Appendix 4 (see separate document).

Table 14: Examples of sentiment analysis for the admission phase of care.

Sentiment Example Positive > Good: Excellent care and quick diagnosis.

> The clerking was 8 hours after admission but was good quality and a good management plan was put in place, correctly identifying the patient was likely to have COVID-19 pneumonitis.

> Excellent, thorough, holistic care. > Very rapid escalation of care. > Very good care.

Negative > Unfortunately moved wards twice due to error in COVID-19 status, therefore care good not excellent.

> PMH‚ dementia, depression and BPSD, type II diabetes mellitus, epilepsy, radical mastectomy, high choking risk, recent admission with UTI, long-term catheter.

> Presented with 2-week history of cough, fever, worsening shortness of breath. > Admitted generally unwell 2/52 of intermittent cough, now worsening

shortness of breath, cough and diarrhoea. > Unwell for 6/7 with fever, myalgia, fatigue, diarrhoea, and cough.

PMH = past medical history; BPSD = behavioural and psychological symptoms of dementia; UTI = urinary tract infection



Summary and conclusions This study recruited a sample and demographic which was representative of the experience of English NHS acute hospitals during the COVID-19 pandemic in 2020. The study gathered information about the quality of care delivered from 19 organisations in England, which between them served a population of 10.46 million people. These organisations collectively looked after over 26,000 patients with COVID-19 in 2020, with over 6,000 patients dying from the disease while in their care.

The key methodology used in the study was based on an established and validated process already used by many clinicians in the retrospective case record analysis of acute hospital deaths. The amount of information released about care quality using the SJR methodology with relatively small sample sizes is extensive and informative, having both quantitative elements and qualitative narrative. The 510 cases reviewed have created a clear description of the care given and provide learning for further improvements.

Multiple quantitative and qualitative analytical methods allowed for triangulation of the results to provide robust evidence from which to draw conclusions and make recommendations. Qualitative narrative analysis also suggested areas requiring further exploration of the variation in care quality.

The study showed categorically that the care provided, as judged by case record review, was adequate, good or excellent in 96.5% of patients in these centres. Rapid assessment was key to the early diagnosis and planning delivered by a consultant-led workforce. When prognosis was poor, rapid decisions about care were usually discussed widely and sensitively and early involvement of palliative care with MDT working was evident.

Care judged to be poor overall was very uncommon and occurred in 3.5% of the total sample of 510. When it did occur, it was related to EOLC issues, nosocomial infections, delays in assessment and the two linked issues of poor communication and poor documentation.

There was variation between hospitals in numbers of patients with COVID-19 admitted and the proportion who died. This was independent of their size, catchment population and age of patients. These differences were likely to be multi-factorial, but further study to understand them will be important to ensure consistent high-quality care in the appropriate setting.

The phrase ‘ceiling of care’ was noted in many individual patient case reviews and is in widespread use in the NHS. It is an inaccurate description of the process it attempts to convey. Describing and documenting the discussions and decisions around escalation of care is more appropriate for practitioners, patients and the public.

Areas of care with significant variation and where there are opportunities for further improvement were:

> EOLC experiences > assessment > documentation and communication > senior review > DNACPR decisions > discharge planning.



These areas revealed both excellent care and care that could have been improved in the majority of centres. This therefore warrants quality improvement approaches within and across organisations, to achieve higher quality more consistently.

There were no appreciable differences in the overall care scores of patients with a learning disability and the other patients, although patients with learning disabilities were younger and those who died were younger. This adds to the information gathered by the LeDeR process, although in this small sample the differences were substantially greater than those found by LeDeR. Although some caution is required because of the low number, the data support the LeDeR findings of poorer EOLC experience for these patients and families. In particular, documentation of discussions and decisions was not clear to the reviewers.

Eight per cent of the patients in this study who did not survive had contracted COVID-19 while receiving treatment in the acute setting for a different reason. Further work to understand nosocomial infection and transmission of COVID-19 is required to protect future patients.

There was an apparent difference in the care quality of those who survived and those who died. This can largely be attributed to the quality of end-of-life care and the acquisition of COVID-19 in hospital. This may be exaggerated by some hindsight bias. The MDT approach and the importance of palliative care were uniformly celebrated and were key in the successful delivery of high-quality care to patients with COVID-19, both in decision making throughout the hospital stay and in the EOLC phase.

Natural language processing algorithms revealed clear statistical differences in themes during the analysis of phases of care. This supported the findings from manual analysis. Further work is required to refine sentiment analysis using this approach, and its potential role in qualitative analysis of case record review.

Recommendations 1 Hospitals and teams that have delivered excellent care during the pandemic should analyse the

factors that have enabled this to happen. They should be encouraged to share the learning from the key findings, locally or nationally to support more consistent quality of care in the future and to raise awareness across the NHS and with the general public.

2 Healthcare professionals and organisations should strive to reduce the variation in care in key areas including:

− end-of-life care in hospital − early assessment in acute care presentations − documentation and communication − senior review and decision making − treatment escalation planning, conversations and documentation − discharge planning − particular focus should be given to reducing variation for those with learning disabilities in

these areas.

3 The guidance and standards published by the RCP covering many of these areas should be implemented and followed more consistently.



4 All NHS organisations are encouraged to pay particular attention to the quality of care, decision making, communication and documentation that is required for vulnerable people, including those with learning disabilities.

5 The NHS should further explore the reasons for the range of hospital mortality between areas during the COVID-19 pandemic.

6 The RCP and Imperial College should further explore the role of natural language processing in the qualitative analysis of structured case record reviews.



Acknowledgements This report was written by Dr Andrew Gibson, clinical lead for patient safety at the Royal College of Physicians (RCP) and deputy medical director and consultant neurologist at Sheffield Teaching Hospitals NHS Foundation Trust.

Thank you to all the colleagues and reviewers from the following 19 NHS organisations who participated in this study and submitted data during the COVID-19 pandemic of 2020–21. Without your input and support this report would not have been possible.

> Barnsley Hospitals NHS Foundation Trust > Bolton Hospital NHS Foundation Trust > Brighton and Sussex University Hospital NHS Foundation Trust > Croydon Health Services NHS Trust > East Kent Hospitals University NHS Foundation Trust > Epsom and St Helier NHS Trust > Guy’s and St Thomas’ NHS Foundation Trust > King’s College Hospital NHS Foundation Trust > Leeds Teaching Hospital NHS Trust > Norfolk and Norwich University Hospital NHS Foundation Trust > North Bristol NHS Foundation Trust > North Middlesex University Hospital NHS Trust > North West Anglia NHS Foundation Trust > Royal Berkshire Hospital NHS Trust > Royal Bournemouth Hospital NHS Foundation Trust > Sheffield Teaching Hospitals NHS Foundation Trust > Southport and Ormskirk NHS Trust > University Hospitals Bristol and Weston NHS Foundation Trust > United Lincolnshire Hospitals NHS Trust

We would like to acknowledge the assistance of the following individuals, who commented on the draft:

> Dr John Dean, RCP clinical director for quality improvement and patient safety and deputy medical director, East Lancashire Hospitals NHS Trust

> Dr Emma Redfern, Association of Emergency Medicine and consultant in emergency medicine, University Hospitals Bristol and Weston NHS Foundation Trust

> Dr Jane Youde, RCP clinical director for audit and accreditation and consultant in elderly care medicine, University Hospitals of Derby and Burton NHS Foundation Trust

The RCP would like to acknowledge the support in the quantitative analysis by the Association of Professional Healthcare Analysts (APHA) and specifically their chief operating officer Rony Arafin in the preparation of the graphics content of the quantitative analysis section of the report.

In addition, the RCP would like to thank the data team at the Institute of Global Health Innovation at Imperial College London, working independently via Imperial Consultants, for their invaluable contribution to the natural language processing and thematic analysis of the modified SJR forms.

Finally, we would like to acknowledge support for the study and report preparation from the RCP’s COVID-19 Appeal funding.



References 1 WHO Coronavirus (COVID-19) Dashboard. https://covid19.who.int

2 Heslop P. Deaths from COVID-19 reviewed as part of the LeDeR programme. April 2020. https://www.bristol.ac.uk/media-library/sites/sps/leder/Summary%20of%20findings%2050%20LeDeR%20reveiws%20of%20deaths%20related%20to%20COVID19.pdf [Accessed 23 June 2021].

3 BBC News. COVID-19 ‘do not resuscitate’ decisions complaints up. 3 December 2020. www.bbc.co.uk/news/health-55163009 [Accessed 23 June 2021].

4 Hutchinson A, Coster JE, Cooper KL et al. A structured judgement method to enhance mortality case note review: development and evaluation. BMJ Qual Saf 2013;22:1032–40.

5 Royal College of Physicians. National Mortality Case Record Review annual report 2018. London: RCP, 2018.

6 Serrano-Guerrero J, Olivas JA, Romero FP et al. Sentiment analysis: A review and comparative analysis of web services. Information Sciences 2015;311:18–38.

7 Office for National Statistics. www.ons.gov.uk/peoplepopulationandcommunity/population andmigration/populationestimates/bulletins/annualmidyearpopulationestimates/mid2019estimates#population-growth-in-england-wales-scotland-and-northern-ireland. 24 June 2020.

8 Docherty AB, Harrison EM, Green CA et al. Features of 20,133 UK patients in hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study. BMJ 2020:22;369:m1985.

9 Hogan H, Zipfel R, Neuburger J et al. Avoidability of hospital deaths and association with hospital-wide mortality ratios: Retrospective case record review and regression analysis. BMJ 2015 Jul 14;351:h3239.

10 Roberts AP, Morrow G, Walkley M et al. From research to practice: results of 7300 mortality retrospective case record reviews in four acute hospitals in the North-East of England. BMJ Open Quality 2017;6:e000123.

11 Oliver D. Could we do better on hospital acquired COVID-19 in a future wave? BMJ 2021;372:n70.

12 Healthcare Safety Investigation Branch. COVID-19 transmission in hospitals: management of the risk – a prospective safety investigation. Independent report. HSIB, 2020.

13 Fischhoff B. Hindsight Foresight: The effect of outcome knowledge on judgment under uncertainty. J Exp Psychol Human Percept Perform 1975 1:288–99.

14 Henriksen K and Kaplan H. Hindsight bias, outcome knowledge and adaptive learning. Qual Safe Health Care 2003 12 (supp 11) ii46-ii50.

15 Hosking FJ, Carey IM, Shah SM et al. Mortality among adults with intellectual disability in England: comparisons with the general population. Am J Pub Health 2016;106:1483–90.

https://covid19.who.int/

https://www.bristol.ac.uk/media-library/sites/sps/leder/Summary%20of%20findings%2050%20LeDeR%20reveiws%20of%20deaths%20related%20to%20COVID19.pdf



http://www.bbc.co.uk/news/health-55163009

http://www.ons.gov.uk/peoplepopulationandcommunity/population%0bandmigration/populationestimates/bulletins/annualmidyearpopulationestimates/mid2019estimatespopulation-growth-in-england-wales-scotland-and-northern-ireland





16 Public Health England. Deaths of people identified as having learning disabilities with COVID-19 in England in the spring of 2020. PHE gateway number GW-1687. London: PHE, 2020.

17 NHS England. Learning from lives and deaths – people with a learning disability and autistic people (LeDeR) policy 2021. Version 1. 23 March 2021.

Supported by:

Caring for hospital patients with COVID-19Quality of care in England examined by case record review

www.rcp.ac.uk/caring-for-hospital- patients-with-COVID-19

Royal College of Physicians11 St Andrews PlaceRegent’s ParkLondon NW1 4LE

© Royal College of Physicians, 2021

http://www.rcp.ac.uk/caring-for-hospital- patients-with-COVID-19

http://www.rcp.ac.uk/caring-for-hospital- patients-with-COVID-19

caring for hospital patients with covid-19 quality of care

Documents