ABSTRACT

Objective:

In this study, we investigated whether scoring systems determine COVID- 19 severity.

Materials and Methods:

COVID-19 patients hospitalized between 01.09.2020 and 31.04.2021 were retrospectively assessed. The National Early Warning Score (NEWS), Modified Early Warning Score (MEWS), Rapid Emergency Medicine Score (REMS), Quick Sequential Organ Failure Assessment Score (q-SOFA), CURB-65, MuLBSTA, and ISARIC 4C scores on admission day were calculated. Scoring systems’ ability to predict mechanical ventilation (MV) need, intensive care unit (ICU) admission, and 30-day mortality were assessed.

Results:

A total of 292 patients were included; 137 (46.9%) were female, and the mean age was 62.5±15.4 years. 69 (23.6%) patients required ICU admission, 45 (15.4%) needed MV, and 49 (16.8%) died within 30 days. No relationship was found between qSOFA and MV need (p=0.167), but a statistically significant relationship was found between other scoring systems and MV need, ICU admission, and 30-day mortality (p<0.05). ISARIC-4C (optimal cut-off >5.5) and NEWS (optimal cut-off >3.5) had the highest area under the curve in ROC curve analyses, whereas qSOFA had the lowest.

Conclusion:

The severity of COVID-19 could be estimated by using these scoring systems, especially ISARIC-4C and NEWS, at the first admission. Thus, mortality and morbidity would be reduced by making the necessary interventions earlier.

Keywords:

COVID-19, ISARIC-4C, mortality, NEWS, scoring systems

Introduction

The coronavirus disease-2019 (COVID-19) has been diagnosed in over 750 million people, and more than 6.8 million people have died due to this disease to date (1). The disease can be asymptomatic or mild with a flu-like syndrome. However, in some cases, it progresses more severely, and pneumonia and acute respiratory distress syndrome (ARDS) can be seen (2). In severe cases, the patient may require mechanical ventilation, admission to the intensive care unit (ICU), and even die. Many studies have examined the correlation between the severity of COVID-19 and markers such as blood type (3), blood inflammation and coagulation biomarkers, and viral load (4). In addition, it is reported that various scoring systems can predict worsening and mortality in COVID-19 patients (5-10). We aimed to investigate whether the scoring systems that can be easily calculated during the emergency admissions of COVID-19 patients determine the requirement for mechanical ventilation, ICU admission, and mortality that may occur in the follow-up of the patients.

Materials and Methods

The research is a single-center, retrospective descriptive study. Patients aged 18 years and over and hospitalized in the infectious diseases clinic and pulmonary diseases clinic with a diagnosis of COVID-19 confirmed by positive severe acute respiratory syndrome-coronavirus-2 polymerase chain reaction between 01.09.2020 and 31.04.2021 in a secondary care hospital were included in our study. The patients’ epidemiological data, chronic diseases, clinical signs, laboratory values detected at the emergency admission, and outcomes were evaluated retrospectively from the patient files. National early warning score (NEWS), modified early warning score (MEWS), rapid emergency medicine score (REMS), quick sequential organ failure assessment score (q-SOFA), CURB-65, MuLBSTA and ISARIC-4C scores were calculated using MDCalc online calculator (https://www.mdcalc.com) at admissions to the hospital (Table 1). The primary endpoint of the study was 30-day mortality. Secondary endpoints were the need for mechanical ventilation and ICU admission.

Statistical Analysis

The statistics of the study were made with the IBM Statistical Package for the Social Sciences (SPSS) Version 22.0 (Armonk, NY: IBM Corp) program. Analytical tests (Kolmogorov-Smirnov/Shapiro-Wilk) were used to check variables for normal distribution. Descriptive analyses were presented using means (± standard deviation) for the normally distributed variables and medians (minimum-maximum) for the non-normally distributed. The Mann-Whitney U test was used to evaluate the association between scoring systems and the endpoints of the study since none of the scoring systems were normally distributed. The capacity of scoring systems in predicting the need for mechanical ventilation, ICU admission, and 30-day mortality were analyzed using receiver operating characteristic (ROC) curve analysis. Significant cut-off values were dedicated, and the sensitivity and specificity values were presented. A power analysis was conducted with a power of 95%, a margin of error of 0.05, and an effect size of 0.8, using the G*Power 3.1.9.2 program. The analysis revealed that a minimum sample size of 108 and 22 participants for groups was required to achieve adequate statistical power. A p-value of less than 0.05 was considered statistically significant.

Results

The data of 445 patients followed up due to COVID-19 within the specified date range were analyzed, and 153 of them did not meet the research criteria due to missing data. Thus, 292 patients were included in the study. The mean age was 62.5±15.4 years, and 137 patients (46.9%) were female.

During the follow-up of the patients, 69 (23.6%) required ICU admission, 45 (15.4%) needed mechanical ventilation, and 49 (16.8%) died within 30 days. The median values of the scoring systems and the distribution of these values according to the outcomes are shown in Table 2. While no statistically significant relationship was found between q-SOFA and the need for mechanical ventilation (p=0.167), a statistically significant relationship was found between all scoring systems except this one and the need for mechanical ventilation, ICU admission, and 30-day mortality (Table 2). When the ROC curve was examined for the outcomes, ISARIC-4C (0.919, 0.974, and 0.918, respectively) and NEWS (0.785, 0.735, and 0.759, respectively) scores were found to have the highest area under the curve (AUC), while q-SOFA (0.543, 0.556, and 0.580, respectively) have the lowest (Figure 1, Figure Table 3). The optimal cut-off values determined for outcomes were found to be >5.5 in the ISARIC-4C score and >3.5 in the NEWS. The percentages of sensitivity and specificity according to the determined optimal cut-off values are shown in Table 3.

Discussion

COVID-19 can be presented with a wide spectrum from asymptomatic to severe disease which may result in death. It is important to be able to predict how the prognosis will progress at the first admission of patients. In our study, the performance of scoring systems, which can be easily calculated during the first admission of COVID-19 patients, to determine the requirement for mechanical ventilation, ICU admission, and 30-day mortality was examined. Especially in patients with ISARIC-4C score >5.5 and NEWS >3.5, COVID-19 disease was found to be more severe, while CURB-65 and MuLBSTA scores had the lowest performance.

During the course of COVID-19, the need for mechanical ventilation with endotracheal intubation may develop due to ARDS (5). Similar to our study, in determining the requirement for mechanical ventilation in COVID-19 patients, Ocho et al. (6) reported that ISARIC-4C (AUC =0.85) was better than CURB-65 (AUC =0.82) and q-SOFA (AUC =0.67), and in another study (7), NEWS (AUC =0.69) was better than q-SOFA (AUC =0.61). Kuroda et al. (8) found that the ISARIC-4C predicts the composite outcome of the need for mechanical ventilation and mortality better than REMS in COVID-19 patients. Chang et al. (9) reported that the detection of NEWS >7 at the first admission to the hospital can determine the need for mechanical ventilation with 72.3% sensitivity and 92.5% specificity. However, it has been reported that the MuLBSTA score (AUC =0.836) is better than CURB-65 and q-SOFA in determining the need for mechanical ventilation (10). In our study, ISARIC-4C [AUC =0.919, 95% confidence interval (CI) 0.887-0.951] and NEWS (AUC =0.785, 95% CI 0.716-0.854) were the best performing scores in line with the literature in demonstrating the requirement for mechanical ventilation of COVID-19 patients while q-SOFA and MuLBSTA performed poorly.

Severe COVID-19 patients may need to be admitted to the ICU for close monitoring and supportive treatment. Studies are reporting that especially the NEWS score is good at predicting ICU admission (11, 12). In a study that compares scoring systems in COVID-19 patients, the NEWS (AUC =0.73) showed the best performance for predicting ICU admission, but good results were not obtained in the q-SOFA, CURB-65, and REMS scores (11). In another study, early warning scores were evaluated and it was reported that the NEWS (AUC =0.783) was more successful in predicting ICU hospitalization within 7 days compared to MEWS, REMS, and q-SOFA scores (12). However, unlike our study, it was reported that CURB-65 (AUC =0.898) was better than ISARIC-4C (AUC =0.797) (13) and MuLBSTA was better than CURB-65 and q-SOFA (10) in predicting ICU admission. In our study, the most successful scores in predicting ICU admission were ISARIC-4C (AUC =0.974, 95% CI 0.959-0.989), NEWS (AUC =0.735, 95% CI 0.667-0.803) and REMS (AUC =0.694, 95% CI 0.626-0.763) while q-SOFA did not show the expected performance.

COVID-19 may have a severe course and be mortal due to reasons such as pneumonia, sepsis, ARDS, and pulmonary thromboembolism (2, 14). It is crucial to identify these patients in the early period for the chance to prevent mortality. Similar to our findings, previous research has shown that the ISARIC-4C and NEWS scores are reliable indicators of mortality in COVID-19 patients (7, 8, 15-17).However, studies are reporting that REMS is better than the q-SOFA, NEWS, MEWS, and CURB-65 scores (11, 12), and CURB-65 is better than the ISARIC-4C (13) in the prediction of mortality. Moreover, MEWS, CURB-65, and q-SOFA scores have also been reported to be successful in predicting mortality (18, 19). Kalani et al. (20) reported that MuLBSTA (AUC =0.832) and CURB-65 (AUC =0.809) scores performed well in predicting 30-day mortality. In our study, ISARIC-4C (AUC =0.918, 95% CI 0.881-0.955), NEWS (AUC =0.759, 95% CI 0.684-0.833), and REMS (AUC =0.756, 95% CI 0.688-0.825) scores were found to be reliable predictors of 30-day mortality, but the q-SOFA did not show promising results.

Our research has limitations. First of all, it is a retrospective study. Secondly, other factors that may cause the need for mechanical ventilation, ICU admission, and mortality such as co-infections were not investigated.

Conclusion

Especially ISARIC-4C and NEWS scores showed high performance in predicting the requirement for mechanical ventilation, ICU admission, and 30-day mortality, but good results were not obtained in q-SOFA. With the early use of these scoring systems in COVID-19 patients, it will be possible to distinguish patients with a risk of clinical worsening. In this way, it was thought that necessary interventions could be made earlier and a decrease in mortality rate could be achieved.

Ethics

Ethics Committee Approval: Ethics committee approval of the study was received from the Non-invasive Clinical Research Ethics Committee of Recep Tayyip Erdoğan University Faculty of Medicine on 19/08/2021 with decision number 2021/149.

Informed Consent: Retrospective study.

References

World Health Organization WHO Coronavirus Disease (COVID-19) Dashboard. Available at:https://covid19.who.int/. Accessed March 7, 2023.

Elezkurtaj S, Greuel S, Ihlow J, Michaelis EG, Bischoff P, Kunze CA, et al. Causes of death and comorbidities in hospitalized patients with COVID-19. Sci Rep 2021;11:4263.

Ozkarafakili MA, Gareayaghi N, Kara ZMY. Relationship Between ABO Blood Types and Coronavirus Disease 2019 Severity. Sisli Etfal Hastan Tip Bul 2022;56:1-20.

Papadopoulou G, Manoloudi E, Repousi N, Skoura L, Hurst T, Karamitros T. Molecular and Clinical Prognostic Biomarkers of COVID-19 Severity and Persistence. Pathogens 2022;11: 311.

Grasselli G, Cattaneo E, Florio G, Ippolito M, Zanella A, Cortegiani A, et al. Mechanical ventilation parameters in critically ill COVID-19 patients: a scoping review. Crit Care 2021;25:115.

Ocho K, Hagiya H, Hasegawa K, Fujita K, Otsuka F. Clinical Utility of 4C Mortality Scores among Japanese COVID-19 Patients: A Multicenter Study. J Clin Med 2022;11:821.

Suresh S, Tiwari A, Mathew R, Bhaskararayuni J, Sahu AK, Aggarwal P, et al. Predictors of mortality and the need of mechanical ventilation in confirmed COVID-19 patients presenting to the emergency department in North India. J Family Med Prim Care 2021;10:542-9.

Kuroda S, Matsumoto S, Sano T, Kitai T, Yonetsu T, Kohsaka S, et al. External validation of the 4C Mortality Score for patients with COVID-19 and pre-existing cardiovascular diseases/risk factors. BMJ Open 2021;11:e052708.

Chang MC, Kim TU, Park D. National early warning score on admission as risk factor for invasive mechanical ventilation in COVID-19 patients: A STROBE-compliant study. Medicine (Baltimore) 2021;100:e25917.

Yeşiltaş S, Meriç Koç M, Karataş A, Tepe G, Pasin Ö. Investigation of the predictive value of MuLBSTA score in predicting critical clinical outcomes in hospitalized patients with severe acute respiratory syndrome-coronavirus-2 pneumonia. Anadolu Kliniği Tıp Bilimleri Dergisi 2022;27:310-7.

Accordino S, Sozzi F, Canetta C. Performance analyses of prognostic scores in critical COVID-19 patients: think outside the numbers. Ann Med 2022;54:1906-7.

Covino M, Sandroni C, Santoro M, Sabia L, Simeoni B, Bocci MG et al. Predicting intensive care unit admission and death for COVID-19 patients in the emergency department using early warning scores. Resuscitation 2020;156:84-91.

Doğanay F, Ak R. Performance of the CURB-65, ISARIC-4C and COVID-GRAM scores in terms of severity for COVID-19 patients. Int J Clin Pract 2021;75: e14759.

Coskun N, Altinay M, Turk HS, Peker N, Islamoglu S, Cinar AS, et al. The effect of High-Dose Vitamin C Treatment for Acute Respiratory Failure Due to Coronavirus Disease Pneumonia on Mortality and Length of Intensive Care Stay: A Retrospective Cohort Study. Sisli Etfal Hastan Tip Bul 2023;57:25-32.

Covino M, De Matteis G, Burzo ML, Russo A, Forte E, Carnicelli A, et al. Predicting in-hospital mortality in COVID-19 older patients with specifically developed scores. J Am Geriatr Soc 2021;69:37-43.

Vedovati MC, Barbieri G, Urbini C, D'Agostini E, Vanni S, Papalini C, et al. Clinical prediction models in hospitalized patients with COVID-19: A multicenter cohort study. Respir Med 2022;202:106954.

Sungurtekin H, Sargın F, Kılınç M, Kahramanoğlu M, Akbulut M, Ayoğlu F, et al. APACHE II, SOFA, ISARIC/WHO 4C Mortalite Skoru ve CO-RADS’ın Kritik Coronavirüs 2019 (Covid-19) Hastalarının Mortalite Tahmininde Kullanımı. J Turk Soc Intens Care 2023;21:100-9.

Aygun H, Eraybar S. The role of emergency department triage early warning score (TREWS) and modified early warning score (MEWS) to predict in-hospital mortality in COVID-19 patients. Ir J Med Sci 2022;191:997-1003.

Wang L, Lv Q, Zhang X, Jiang B, Liu E, Xiao C, et al. The utility of MEWS for predicting the mortality in the elderly adults with COVID-19: a retrospective cohort study with comparison to other predictive clinical scores. PeerJ 2020;8:e10018.

Kalani N, Tavasolian M, Dehghani K, Mousavi SR, Ghanbarzadeh E, Shakeri M, et al. Prognostic value of intensive care scores concerning the prediction of 30-day mortality in COVID-19. Journal of Emergency Practice and Trauma 2023;9:19-24.

Predicting Mechanical Ventilation, Intensive Care Unit Admission, and Mortality in COVID-19 Patients: Comparison of Seven Different Scoring Systems