The main finding of the study is that the APACHE II score was the most accurate and had the best discrimination at predicting mortality risk in COVID-19 patients treated in the ICU. However, the best calibration was observed when the 4C Mortality Score was added to the model. Therefore, the APACHE II and 4C Mortality Score independently predict mortality risk and can be used concomitantly.
One of the main findings of the study is that conventional ICU mortality risk scores perform quite well in COVID-19 patients. In our study, the mean values of the APACHE II, SAPS II and SOFA scores are comparable with other reports. The overall mean APACHE II score in our study was 12, which is comparable to the 12.87 reported from India but lower than the ones reported from Sao Paolo (16.7) and Pakistan (20.84) [14,15,16]. Furthermore, in our study, the APACHE II score prevailed as the most accurate one with the highest AUC of all the scores (0.772) [17, 18]. These results correspond with the report from Sao Paolo (AUC 0.8). However, the overall accuracy is slightly lower than expected and reported in the literature [19]. Furthermore, the SMRs are several times higher than expected in these patients. This can be explained by the pathophysiology of the COVID-19 disease, which affects several organ systems far more extensively (i.e., respiratory and coagulation) in the beginning of the disease. Thus, the conventional scores, even when giving a maximum score in these dimensions, may under evaluate the overall mortality risk of these patients.
Secondly, the 4C Mortality Score fits into the risk prediction model with the APACHE II score for ICU patients. In studies comparing the risk scores developed specifically for COVID-19 patients, the 4C Mortality Score was the most accurate, with AUCs of 0.799 and 0.774, and 0.754 in our study. This score has a more extensive evaluation of the comorbidities of patients than conventional ICU risk scores, which tend to rely on stratifying the on-the-spot evaluation of the organ systems. In our study, the combination of the APACHE II and 4C Mortality Score increased the calibration of the risk determination model in the regression analysis [12, 18]. Despite this, the R2 value of 0.358 in the final model was only satisfactory when predicting the mortality in this group. It is obvious that the clinical course in the ICU and the application of mechanical ventilation, renal replacement therapy and other treatment options are major contributors to the outcome. Thus, further studies should be done either to elucidate more risk factors or to define the key moments in the treatment of this specific population.
It is important to discuss the potential clinical implementation of our findings. In the case of the pandemic and the overload of patients, the most valuable feature of the mortality score is good discriminative performance and pragmatic identification of the patients who are likely to die and will not benefit from treatment. Thus, the 4C Mortality Score is a perfect tool in the emergency department (ED). However, when evaluating the mortality of COVID-19 patients in the ICU, a more precise tool is needed. For example, since the 4C Mortality Score was developed for triage in the ED, a noninvasive oxygenation parameter (SpO2) was chosen to evaluate respiratory function. Pulse oximetric saturation has a good correlation with PaO2 in the range of 80–100%. However, critically ill COVID-19 patients usually stay in the ward until SpO2 levels drop below 80%. The accuracy of SpO2 drops when the SpO2 level is below 80% [20]. Moreover, in the case of the APACHE II score, it has a potentially better evaluation of chronic respiratory failure because of the inclusion of base excess. Furthermore, SpO2 strongly depends on blood flow, pulsatility and microcirculatory disturbances (such as microthrombosis), which can affect the accuracy of SpO2. Moreover, the APACHE II and SAPS II scores include a lot of parameters that are usually normal on admission in COVID-19 patients, but they may deteriorate during the clinical course of these patients, leading to high prognostic value when admitting patients to the ICU. Thus, we conclude that there is a benefit to combine the use of these scores. We suggest using the 4C Mortality Score when admitting patients to the hospital and to consider it when accepting the patient to the ICU, alongside the calculation of the APACHE II score.
This study is retrospective. Therefore, all the assumptions should be regarded as associations rather than as causations. Furthermore, it is important to consider the sample size of our study and keep in mind that, to get more generalizable results, more patients should be included in the study. There is a slight difference in the age of the patients in the study when compared with the literature. The mean age of the patients in our study was 61.32 years, which is an average number compared to several other studies with mean ages of 50, 51.26 and 74 years [14, 16, 18]. However, the proportion of patients over 70 years of age is quite large, almost 30%. Increasing age is known to be associated with higher mortality risk and poor outcome. This and other differences between the population of our study and the studies that validated the risk scores (ethnic, demographic, cultural and economic conditions of the patients) should also be considered.
This was a study with a pragmatic approach, having its design oriented towards clinical decision-making. If a patient in the ED met the criteria for hospitalization, he/she was evaluated with the 4C Mortality Score, independently whether he/she was hospitalized on a general ward or directly to the ICU. If the patient was transferred to the ICU, either directly or later, he/she was evaluated with ICU risk scores (APACHE II, SOFA and SAPS II). Therefore, the study design has a potential for bias, since the patient from the general ward may deteriorate and be transferred in a more grave state as compared to the state he/she was hospitalized and evaluated with the 4C Mortality Score. However, the 4C Mortality Score was not developed to use in patients that are already hospitalized. Thus, to offer a clinician a pragmatic way of risk evaluation, we designed this study with evaluations at two points in time, indeed, in some cases, occurring at the same time, when the patient is being admitted directly to the ICU and, in some cases, at two different points in time.