Skip to main content
SearchLogin or Signup

Review 2: "The utility of established prognostic scores in COVID-19 hospital admissions: a multi-centre prospective evaluation of CURB-65, NEWS2, and qSOFA"

This robust analysis is novel and of high interest for the medical community. This study informs how new prognostic scores should be created to more accurately guide clinical decision-making in patients with COVID-19.

Published onAug 25, 2020
Review 2: "The utility of established prognostic scores in COVID-19 hospital admissions: a multi-centre prospective evaluation of CURB-65, NEWS2, and qSOFA"
1 of 2
key-enterThis Pub is a Review of
The utility of established prognostic scores in COVID-19 hospital admissions: a multicentre prospective evaluation of CURB-65, NEWS2, and qSOFA
Description

INTRODUCTION The COVID-19 pandemic is ongoing yet, due to the lack of a COVID-19 specific tool, clinicians must use pre-existing illness severity scores for initial prognostication. However, the validity of such scores in COVID-19 is unknown. METHODS The North West Collaborative Organization for Respiratory Research (NW-CORR) performed a multi-centre prospective evaluation of adult patients admitted to hospital with confirmed COVID-19 during a two-week period in April 2020. Clinical variables measured as part of usual care at presentation to hospital were recorded, including the CURB-65, NEWS2, and qSOFA scores. The primary outcome of interest was 30-day mortality. RESULTS Data were collected for 830 people with COVID-19 admitted across 7 hospitals. By 30 days, a total of 300 (36.1%) had died and 142 (17.1%) had been in ICU. All scores underestimated mortality compared to their original validation in non-COVID-19 populations, and overall prognostic performance was generally poor. Among the 9low risk9 categories (CURB-65<2, NEWS2<5, qSOFA<2) 30-day mortality was 16.7%, 32.9% and 21.4%, respectively. Multivariable logistic regression identified features of respiratory compromise rather than circulatory collapse as most relevant prognostic variables. CONCLUSION All existing prognostic scores evaluated here underestimated adverse outcomes and performed sub-optimally in the COVID-19 setting. New prognostic tools including a focus on features of respiratory compromise rather than circulatory collapse are needed. We provide a baseline set of variables which are relevant to COVID-19 outcomes and may be used as a basis for developing a bespoke COVID-19 prognostication tool.

RR:C19 Evidence Scale rating by reviewer:

  • Strong. The main study claims are very well-justified by the data and analytic methods used. There is little room for doubt that the study produced has very similar results and conclusions as compared with the hypothetical ideal study. The study’s main claims should be considered conclusive and actionable without reservation.

***************************************

Review:

BACKGROUND

The COVID-19 pandemic placed a tremendous strain on global healthcare systems as many hospitals at pandemic epicenters reached inpatient and intensive care unit (ICU) capacity despite considerable public health measures to “flatten the curve”. Multiple facets of medical care – from outpatient primary and urgent care to inpatient emergency and critical care – were faced with the task of triaging patients towards home quarantine, hospital admission, ICU escalation, or comfort-focused care. Especially in the early months of the pandemic, these decisions were being made with scarce data regarding the potential risks and benefits of these decisions and either limited or skewed data on the morbidity and mortality associated with SARS-CoV-2. In addition to guidance from the World Health Organization and national public health and medical agencies (such as the Centers for Disease Control and Prevention and National Institutes of Health in the United States of America and the National Health Service in the United Kingdom), healthcare providers likely relied on either their clinical intuition or risk stratification tools that were previously validated for pneumonia (CURB-651), sepsis (qSOFA2), or acute illness more generally (NEWS23,4). However, the utility of these tools for guiding decision-making for COVID-19 is unknown, and the present study by Bradley et al.5 aimed to assess the performance of these clinical scores and their component variables for predicting early death (all-cause mortality within 72 hours of admission) and both all-cause mortality and ICU admission within 30 days of admission.

MAIN FINDINGS

Among a prospective sample of 830 patients admitted to 7 hospitals in northwest England, death occurred in 7.6% of patients within 72 hours of admission and 36.1% within 30 days of admission, and ICU admission occurred in 15.0% of patients within 72 hours of admission and 17.1% within 30 days of admission. The performance of CURB-65, NEWS2, and qSOFA scores for prediction of 72-hour and 30-day mortality and 30-day ICU admission was measured using the area under the curve (AUC) of receiver operator characteristics (ROC) curves, sensitivity and specificity (Table 1). Overall, the performance of these pre-COVID-19 clinical risk scores at their traditional thresholds was only modest (AUC 0.56-0.78), being the least accurate in predicting ICU admission. The authors also developed new prognostic scores (CUCA-SF, CUCAF-SR, CAS-FC) from variables independently associated with outcomes in a backwards selection multivariable model (age, Rockwood clinical frailty score, respiratory rate, pulse oximetry, supplemental oxygen requirement, blood urea nitrogen, radiographic pulmonary consolidation, and C-reactive protein) that performed comparatively better (AUC 0.77-0.83).

Table 1. Summary of diagnostic performance of prognostic scores from Bradley et al. (2020)


AUC

Sensitivity

Specificity

CURB-65 (high risk: ≥2)




72-hour mortality

30-day mortality

ICU admission

0.75

0.74

0.63

0.85

0.80

0.37

0.47

0.59

0.41

NEWS2 (high risk: ≥5)




72-hour mortality

30-day mortality

ICU admission

0.78

0.67

0.65

0.92

0.83

0.82

0.31

0.37

0.32

qSOFA (high risk: ≥2)




72-hour mortality

30-day mortality

ICU admission

0.66

0.62

0.56

0.42

0.27

0.11

0.84

0.87

0.80

CUCA-SF (high risk: ≥2)




72-hour mortality

0.82

0.98

0.13

CUCAF-SR (high risk: ≥2)




30-day mortality

0.77

0.98

0.14

CAS-FC (high risk: ≥2)




ICU admission

0.83

0.59

0.88

Abbreviations: AUC, area under the curve.

STUDY REVIEW

Strength of Evidence: Strong

Claims are very well-supported by the data and methods used. Decision-makers should consider the claims in this study actionable without reservation based on the methods and data.

The major claim made by the authors or the presented data is that extant risk stratification scores (CURB-65, NEWS2, qSOFA) offer insufficient predictive value for early clinical deterioration and mortality, and the authors suggest that novel prognostic scores should be created to more accurately guide clinical decision-making in patients with COVID-19. Recent studies have attempted to do just that and have demonstrated reasonable predictive value using measurements that would be readily accessible in clinical practice (Table 2), and the authors also developed predictive scores from clinical factors that were significantly associated with outcomes of interest in COVID-19 patients, which did perform slightly better than predictive tools not specific to COVID-19.

Table 2. Diagnostic performance of diagnostic scores from recently published reports


AUC

Sensitivity

Specificity

CALL score (high risk: ≥10)6

0.91

0.45

0.97

NOCOS calculator (high risk: ≥51.6%)7

0.82

0.89

0.54

qCSI score (high risk: ≥4)8

0.81

0.79

0.79

4C mortality score (high risk: ≥9)9

0.78

0.93

0.41

CALL score for predicting clinical progression included: comorbidities, age, lymphocyte count, and lactate dehydrogenase. NOCOS calculator for predicting 7-day survival included: serum blood urea nitrogen, age, absolute neutrophil count, red cell distribution width, oxygen saturation, and serum sodium. qCSI score for respiratory failure within 24 hours included: respiratory rate, minimum recorded pulse oximetry, and nasal cannula flow rate requirement. 4C mortality score for predicting in-hospital mortality included: age, sex, number of comorbidities, respiratory rate, pulse oximetry on room air, Glasgow coma scale, serum urea, and C-reactive protein. Abbreviations: AUC, area under the curve.

The cumulative evidence from these studies, and others reported in recent systematic reviews of clinical prediction models in COVID-19, suggest that age, quantifications of respiratory compromise (respiratory rate, pulse oximetry nadir, supplemental oxygen requirement, radiographic pulmonary consolidation), measurements of inflammation (C-reactive protein, lactate dehydrogenase, interleukin), and evidence of multiorgan involvement (creatinine and blood urea nitrogen, transaminases and bilirubin) are among the strongest predictors of mortality and clinical deterioration.10,11 What stands out from these studies is the absence of clinical biomarkers more specific to COVID-19 pathophysiology or more reflective of complications increasingly recognized with COVID-19, such as coagulopathy and neurological involvement (stroke, encephalopathy). Further research should be done, and is underway, to more accurately guide healthcare provider decisions regarding diagnosis, need for hospitalization, likelihood of clinical deterioration, and mortality in COVID-19 patients.


REFERENCES

1. Lim WS, van der Eerden MM, Laing R, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58(5):377-382.

2. Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):762-774.

3. Royal College of Physicians. National Early Warning Score (NEWS) 2: Standardising the assessment of acute-illness severity in the NHS. Updated report of a working party. London: 2017.

4. Pimentel MAF, Redfern OC, Gerry S, et al. A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: A multi-centre database study. Resuscitation. 2019;134:147-156.

5. Bradley P, Frost F, Tharmaratnam K, Collaborators N-C, Wootton DG. The utility of established prognostic scores in COVID-19 hospital admissions: a multi-centre prospective evaluation of CURB-65, NEWS2, and qSOFA. medRxiv. 2020.

6. Ji D, Zhang D, Xu J, et al. Prediction for Progression Risk in Patients with COVID-19 Pneumonia: The CALL Score. Clin Infect Dis. 2020.

7. Levy TJ, Richardson S, Coppa K, et al. Development and Validation of a Survival Calculator for Hospitalized Patients with COVID-19. medRxiv. 2020.

8. Haimovich A, Ravindra NG, Stoytchev S, et al. Development and validation of the quick COVID-19 severity index (qCSI): a prognostic tool for early clinical decompensation. Ann Emerg Med. 2020.

9. Knight SR, Ho A, Pius R, et al. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score. medRxiv. 2020.

10. Gupta RK, Marks M, Samuels THA, et al. Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: An observational cohort study. medRxiv. 2020.

11. Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ. 2020;369:m1328.

Comments
1
Ev
Ed J van Hezik: Serum H₂S is a prognostic marker of mortality in COVID-19 pneumonia. Insufficient H₂S level at admission or a decreasing level during infection increases the mortality risk significantly in COVID-19 patients according to a recent study.Renieris G et al.(2020) Serum Hydrogen Sulfide and outcome association in pneumonia by the Sars-CoV-2 Coronavirus. Shock Injury,Inflammation and Sepsis,May 18,2020 doi:10.1097/shk.0000000000001562 HOST FACTOR H₂S CAN BE REACTIVATED Please see: https://www.researchgate.net/publication/341650273_''_Hidden_in_plain_sight_-HS_a_Host_Factor_in_COVID-19_''
Ev
Ed J van Hezik: High dose intravenous NAC is very effective in COVID-19 ,even if treated with ventilator machine and ECMO together ;Just snatched away from the gates of heaven….. -Ibrahim, H., Perl, A., Smith, D., Lewis, T., Kon, Z., Goldenberg, R., Yarta, K., Staniloae, C., & Williams, M (2020),Therapeutic blockade of inflammation in severe COVID-19 infection with intravenous n-acetylcysteine.Clinicalimmunology(Orlando, Fla.), 219, 108544. Advance online publication. https://doi.org/10.1016/j.clim.2020.108544