Skip to main content
Download PDF

Comprehensive statistical analysis reveals significant benefits of COVID-19 vaccination in hospitalized patients: propensity score, covariate adjustment, and feature importance by permutation

BMC Infectious Diseases volume 24, Article number: 1052 (2024) Cite this article

Abstract

Background

COVID-19 vaccines effectively prevent infection and hospitalization. However, few population-based studies have compared the clinical characteristics and outcomes of patients hospitalized for COVID-19 using advanced statistical methods. Our objective is to address this evidence gap by comparing vaccinated and unvaccinated patients hospitalized for COVID-19.

Methods

This retrospective cohort included adult COVID-19 patients admitted from March 2021 to August 2022 from 27 hospitals. Clinical characteristics, vaccination status, and outcomes were extracted from medical records. Vaccinated and unvaccinated patients were compared using propensity score analyses, calculated based on variables associated with vaccination status and/or outcomes, including waves. The vaccination effect was also assessed by covariate adjustment and feature importance by permutation.

Results

From the 3,188 patients, 1,963 (61.6%) were unvaccinated and 1,225 (38.4%) were fully vaccinated. Among these, 558 vaccinated individuals were matched with 558 unvaccinated ones. Vaccinated patients had lower rates of mortality (19.4% vs. 33.3%), invasive mechanical ventilation (IMV-18.3% vs. 34.6%), noninvasive mechanical ventilation (NIMV-10.6% vs. 22.0%), intensive care unit admission (ICU-32.0% vs. 44.1%) vasoactive drug use (21.1% vs. 32.6%), dialysis (8.2% vs. 14.7%) hospital length of stay (7.0 vs. 9.0 days), and thromboembolic events (3.9% vs.7.7%), p < 0.05 for all. Risk-adjusted multivariate analysis demonstrated a significant inverse association between vaccination and in-hospital mortality (adjusted odds ratio [aOR] = 0.42, 95% confidence interval [CI]: 0.31–0.56; p < 0.001) as well as IMV (aOR = 0.40, 95% CI: 0.30–0.53; p < 0.001). These results were consistent in all analyses, including feature importance by permutation.

Conclusion

Vaccinated patients admitted to hospital with COVID-19 had significantly lower mortality and other severe outcomes than unvaccinated ones during the Delta and Omicron waves. These findings have important implications for public health strategies and support the critical importance of vaccination efforts, particularly in low-income countries, where vaccination coverage remains suboptimal.

Peer Review reports

Background

SARS-CoV-2 has infected more than 774 million patients, with more than seven million deaths worldwide [1]. One landmark of the COVID-19 pandemic was the effort to develop and distribute effective vaccines against the virus. Vaccination campaigns have been a key component of public health strategies worldwide to mitigate the impact of the COVID-19 pandemic and are one of the key factors responsible for controlling the disease [2]. Over 13.5 billion doses of immunizers have been administered worldwide, 67% of the total population is vaccinated with a complete primary series of COVID-19 vaccines, and only 32% receive a booster dose [1].

Since the end of 2020, several vaccines have been developed and approved for use [3]. Available COVID-19 vaccines effectively prevent symptomatic SARS-CoV-2 infection, COVID-19-related hospitalization, and death [4]. The benefit has been sustained even with the frequent emergence of new genetic variants of SARS-Cov-2 throughout the pandemic [5,6,7,8].

Brazil faced significant challenges during the COVID-19 pandemic, emerging as one of the most heavily affected countries globally in the number of cases and deaths [9]. In January 2021, the country began a national vaccination campaign against COVID-19. Since then, over 518 million doses of vaccines have been distributed and administered nationwide [10]. Currently, approximately 80.2% of Brazilians over six months of age are fully vaccinated. A particularity of the national vaccination campaign was the use of three platforms of vaccines and four different immunizers: inactivated virus vaccine (CoronaVac®), viral vector vaccine (Covishield; ChAdOx1/AstraZeneca® and Ad26.COV2.S/Jansen®) and messenger RNA vaccine (BNT162b2/Pfizer®), which can be mixed after completing the basic immunization [11].

Despite the available evidence, few population-based studies have compared the clinical characteristics and outcomes of patients hospitalized for COVID-19 concerning vaccination status. This knowledge gap was even greater when an attempt was made to match vaccinated versus unvaccinated individuals to control for confounders [12, 13]. This is of utmost importance, as most countries, including Brazil, use age and comorbidities to prioritize patients for vaccination and booster doses. Observational studies on this topic identified as a limitation the challenge of comparing outcomes between heterogeneous groups, particularly concerning age and comorbidities [14].

Therefore, this study aimed to compare the clinical characteristics and outcomes of vaccinated and unvaccinated COVID-19 in hospital patients, from 27 hospitals across five Brazilian states during the Delta and Omicron waves, using advanced statistical methods. In this study, we sought to provide the first large-scale evaluation of the effectiveness of COVID-19 vaccines implemented by the Brazilian vaccination program during the 2021–2022 period.

Materials and methods

Study design, setting, and subjects

This was a retrospective multicenter cohort study, a sub-study of the Brazilian COVID-19 registry, described in detail previously [15]. It was conducted in 27 hospitals from 14 cities in five states (Bahia, Minas Gerais, Rio Grande do Sul, Santa Catarina, and São Paulo) (Supplementary Table S1).

The study included adult patients (aged ≥ 18 years) with a laboratory-confirmed diagnosis of COVID-19 [16] admitted to the participating hospitals from March 1, 2021, to August 31, 2022. Exclusion criteria were pregnancy, age under 18 years, manifestation of COVID-19 after hospitalization (where patients were admitted for reasons other than COVID-19), discharge within 24 h, transfer to non-participating hospitals, and incomplete vaccination (less than two doses).

The study period comprehended patients who had acute COVID-19 during the second (from November 2020 to December 2021) or third (from December 2021 to May 2022) pandemic waves, corresponding to Delta and Omicron variants, respectively, based on the most predominant circulating variant in Brazil at the time [17].

Data collection

The data were collected from medical records by trained health professionals and undergraduate students (Medicine and Nursing), using a prespecified case report form in the Research Electronic Data Capture (REDCap®) database [18, 19]. The database was hosted at the Telehealth Center, University Hospital, Universidade Federal de Minas Gerais [20].

Baseline variables collected included age, sex, underlying comorbidities, preadmission medications, vaccination status, clinical assessment, and laboratory data at hospital presentation. We also collected medications, interventions during hospitalization, and outcomes, as described in detail previously [21].

Vaccination status was determined based on information extracted from the patient’s medical records. We recorded the total number of vaccine doses administered and the type of vaccine received, including all vaccines that were allowed and approved for use in Brazil during the study period. Patients were then categorized into two distinct groups for analysis: “unvaccinated”, which included those who had received 0 doses, and “fully vaccinated”, which included those who had received two or more doses of any vaccine.

Outcomes

The primary outcomes were hospital mortality and the need for invasive mechanical ventilation (IMV). The secondary outcomes were noninvasive mechanical ventilation (NIMV), dialysis, vasoactive drugs, thrombosis (deep vein thrombosis, pulmonary embolism, arterial thrombosis), intensive care unit (ICU) admissions, days in the ICU, days on IMV, and length of hospital stay [22].

Statistical analysis

To account for potential confounding, covariate adjustment, and propensity score (PS) analyses were performed. The PS was estimated using a logistic regression model that incorporated all variables potentially related to the vaccination decision and/or the outcomes: age, sex, hospital of care, comorbidities (hypertension, coronary artery disease, heart failure, atrial fibrillation, stroke, asthma, chronic obstructive pulmonary disease, pulmonary fibrosis, diabetes mellitus, obesity [body mass index > 30 kg/m2], chronic kidney disease, dialysis, rheumatologic disease, HIV, cancer, post-transplant, and cirrhosis), and home medications (anticoagulation, oral corticosteroids, and immunosuppressants) and pandemic waves.

We performed three PS methods: PS matching, inverse probability weighting (IPW), and inclusion of PS as a covariate. PS matching was applied for all outcomes. For the primary outcome only, we additionally performed IPW and included PS as a covariate in the logistic regression models. For each method, logistic regression analyses were conducted to examine the risk of in-hospital death and IMV according to vaccination status. This was done as a crude model and by adjusting for the same variables used to generate the propensity score (a doubly robust approach). Results are expressed as odds ratios (ORs) with 95% confidence intervals (CIs).

In PS matching analysis, unvaccinated patients were searched to find those with the closest PS from the vaccinated group (within 0.16 standard deviations of the logit of the propensity score on a scale from 0 to 1.00) using the MatchIt package in R software. The balance of baseline characteristics between the two groups (unvaccinated vs. fully vaccinated) was evaluated using the absolute standardized mean differences. A standardized mean difference of less than 10% indicates a well-balanced covariate between groups.

Categorical data were presented as absolute frequencies and proportions, and continuous variables were expressed as medians and interquartile ranges (IQRs). Groups were compared using the Mann-Whitney test or t-test for quantitative variables and the Chi-square test or Fisher’s exact test for categorical variables.

To better understand the influence of vaccination on the primary outcomes, we also exploited a machine learning technique based on feature importance by permutation [23]. This technique evaluates the contribution of each variable to the effectiveness of a prediction model for the outcome of interest, calculating the importance of the variable based on the reduction in the predictive capacity of the model when shuffling the values ​​of the variables. This approach allows for the evaluation of both the effect of interactions between variables and the main impact of each characteristic in the model, with values being comparable between different outcomes and directly related to the reduction in predictive performance [24]. The most important variables for each primary outcome were extracted, regardless of whether the variable increased or decreased the probability of the outcome (maximum of ten).

The problem was formulated as a binary classification problem with two classes (outcome and non-outcome). The classifier used was XGBoost [25], with a cutoff above random 50% for each outcome to determine a good accuracy.

All analyses were performed using R software (including tyverse, gtsummary, and MatchIt packages, among others) and Python (including libraries such as scikit-learn, numpy, pandas, and XGBoost, among others). The significance level was set at 0.05 (two-tailed P-value < 0.05).

Results

Overall, 3,188 patients were eligible for the study (Fig. 1). Of these, 1,225 were fully vaccinated, and 1,963 were unvaccinated. Of the 1,225 fully vaccinated patients, CoronaVac® was the most common vaccine (70.0%), followed by AstraZeneca® (24.0%), Pfizer® (5.2%), and others (0.3%). Regarding doses, 64.0% received a complete series (two doses), and 36.0% received one or more booster doses (Supplementary Table S2).

Propensity score matching randomly selected 1,116 patients: 558 fully vaccinated patients and 558 unvaccinated ones (Fig. 1). Demographic characteristics and comorbidities of both unmatched and matched groups are shown in Table 1.

Fig. 1
figure 1

Flowchart of COVID-19 patients included in the study

Full size image

Unmatched unvaccinated vs. vaccinated

Vaccinated patients were significantly older (73 [64–81] years vs. 57 [46–68] years, p < 0.001) and had a higher frequency of different comorbidities, except for obesity (13.9% vs. 24.7%, p < 0.001), when compared to unvaccinated ones (Table 1).

Vaccinated patients had lower rates of IMV (19.4% vs. 26.5%, p < 0.001), NIMV (11.3 vs. 19.9%, p < 0.001), and thromboembolic events (2.9% vs. 7.7%, p < 0.001) compared to unvaccinated ones. Additionally, vaccinated patients had a shorter ICU length of stay (8.0 vs. 9.0 vs. days, p = 0.013). There were no significant differences in mortality or other outcomes assessed (Supplementary Table S3).

Table 1 Baseline demographics and clinical characteristics: COVID-19 unmatched and matched patients
Full size table

Propensity scores matching analysis

After PS matching, the two cohorts were well balanced, except for a higher frequency of chronic kidney disease in vaccinated patients (8.1% vs. 5.0%, p = 0.04). No significant differences were observed in demographic characteristics or other comorbidities. The characteristics between the two groups were comparable, with an absolute standardized mean difference of less than 10%, as shown in Table 1 and Supplementary Figure S4.

At hospital presentation, vaccinated patients had a higher ratio of arterial oxygen partial pressure to a fraction of inspired oxygen ratio (300 [233–379] vs. 285 [198–360], p = 0.021). They also received fewer antibiotics (57.5% vs. 65.9%, p = 0.004) and systemic corticosteroids (86.6% vs. 90.7%, p = 0.03) during hospitalization than unvaccinated ones (Supplementary Table S5).

In-hospital mortality (19.4% vs. 33.3%, p < 0.001) and IMV (18.3% vs. 34.6%, p < 0.001) were less frequent in vaccinated patients. They also had a lower incidence of NIMV (10.6% vs. 22.0%, p < 0.001), dialysis (8.2% vs. 14.7%, p < 0.001), ICU admission (32.0% vs. 44.1%, p < 0.001), thromboembolic events (3.9% vs. 7.7%, p = 0.007), and vasoactive drug usage (21.1% vs. 32.6%, p < 0.001) than unvaccinated ones. Additionally, the length of hospital stay was shorter (7.0 vs. 9.0 days, p < 0.001). There were no statistically significant differences in the duration of IMV and ICU length of stay (Table 2).

Table 2 Outcomes during hospital stay: COVID-19 matcheda patients
Full size table

Advanced statistical analyses

In univariable analysis of the unmatched sample, vaccinated patients had comparable rates of in-hospital mortality with an unadjusted OR of 1.06 (95% CI: 0.89–1.26, p = 0.509). After adjusting for confounding factors using the multivariable regression model, vaccinated patients had a lower risk of in-hospital mortality, with an adjusted odds ratio (aOR) of 0.51 (95% CI: 0.40–0.64, p < 0.001).

In all three-propensity score (PS) analyses performed (PSM, IPW, and PS as a covariate), vaccinated patients consistently demonstrated lower unadjusted odds of mortality. The benefit of vaccination became even more evident after multivariable regression (doubly robust) with adjusted odds ratios (aORs) of 0.42 (95% CI: 0.31–0.56, p < 0.001) in PSM, 0.49 (95% CI: 0.43–0.57, p < 0.001) in IPW, and 0.44 (95% CI: 0.34–0.57, p < 0.001) in PS as a covariate (Fig. 2A; Supplementary Table S6).

Vaccinated patients also had lower rates of IMV in the unmatched sample, both in unadjusted and adjusted analyses. These lower IMV rates in vaccinated patients were confirmed across all PS methods performed (PSM, IPW, and PS as a covariate), in both crude and adjusted analyses (Fig. 2B; Supplementary Table S7).

Fig. 2
figure 2

Effects of vaccination on mortality (A) and IMV (B) in an original unmatched cohort (crude), covariate adjustment, matching (PSM) (crude and doubly robust), IPW (crude and doubly robust) and PS as a covariate (crude and doubly robust). Forest plots on the log scale show unadjusted and multivariable-adjusted odds ratios (ORs; indicated by diamonds) and 95% confidence intervals (CIs; indicated by the horizontal bars). The variables included in the multivariable regression models were age, sex, hospital of care, comorbidities (hypertension, coronary artery disease, heart failure, atrial fibrillation, stroke, asthma, COPD, pulmonary fibrosis, diabetes mellitus, obesity [body mass index > 30 kg/m2], chronic kidney disease, dialysis, rheumatologic disease, HIV, cancer, post-transplant, and cirrhosis), and home medications (anticoagulation, oral corticosteroids, and immunosuppressants). Abbreviations: IMV: invasive mechanical ventilation; PSM: propensity score matching; IPW: inverse probability weighting; PS: propensity score; OR: odds ratio; CI: confidence intervals

Full size image

The accuracy of the XGBoost model used for the permutation technique to estimate feature importance was between 63% and 93%, indicating high effectiveness (Supplementary Table S8). Vaccination was identified as a key variable associated with a decreased likelihood of death and IMV (Fig. 3A and B).

Fig. 3
figure 3

The most important variables for the predictive models of mortality (A) and IMV (B). In the graph, the size of each bar reflects the importance of the variable for classifying instances for the outcome of interest, while the direction of the bar indicates the association of the variable with the outcome - whether the variable worsens or minimizes the outcome. The direction of the bar was determined based on coefficients from a logistic regression trained to predict outcomes. If a variable’s bar is to the right and is red, it means that the variable is associated with an increase in the probability of the outcome occurring. If the bar is to the left and is blue, this indicates that the variable is associated with a decrease in the probability of the outcome. The values on the X-axis are the measures of loss of effectiveness of the model when performing the permutation. Abbreviations: COPD, chronic obstructive pulmonary disease; CKD; HIV, human immunodeficiency virus

Full size image

Discussion

This study utilized advanced statistical methods, including propensity score, covariate adjustment, and machine learning techniques, to adjust for confounding variables within a large cohort of Brazilian inpatients. Our findings consistently demonstrated that vaccinated patients had lower rates of mortality, IMV, NIMV, dialysis, vasoactive drug use, ICU admissions, and shorter hospital lengths of stay.

Consistent with prior research, our pre-matching sample revealed that vaccinated patients were older and had a higher prevalence of multiple comorbidities, variables known to impact COVID-19 prognosis [26,27,28,29]. This imbalance was anticipated due to the prioritization strategy of the Brazilian vaccination campaign [30]. Therefore, PS matching was essential for diminishing these differences between groups. After PS matching, baseline characteristics between the two groups were well balanced, except for a higher frequency of chronic kidney disease among vaccinated patients.

Despite this imbalance, vaccinated patients had lower incidences of all primary outcomes assessed and almost all secondary outcomes, including dialysis. In our primary outcome analysis, matched vaccinated individuals exhibited an absolute difference of 13.9% in mortality, with an adjusted OR (aOR) of 0.42 (95% CI: 0.31–0.56, p < 0.001), and 16.3% in IMV, with an aOR of 0.40 (95% CI: 0.30–0.53, p < 0.001). These benefits were confirmed through inverse probability weighting (IPW) and propensity score as covariate analyses, further reinforcing the protective effect of vaccination.

The feature importance analysis, using machine learning approaches (XGBoost and Feature Importance by Permutation), further underscores the protective effect of vaccination against severe outcomes in hospitalized COVID-19 patients. Specifically, being vaccinated emerged as a key variable associated with a decreased probability of both mortality and the need for IMV. This finding aligns with the results from our PS analyses, which consistently showed lower adjusted odds ratios for death and IMV among vaccinated patients.

Previous studies with hospitalized patients, albeit with smaller sample sizes, have also employed PS techniques to evaluate severe outcomes among vaccinated and unvaccinated COVID-19 inpatients, and have shown various results. An Italian multicenter retrospective cohort study conducted from February 2021 to November 2021 compared 179 vaccinated adults hospitalized with COVID-19 (who received at least one dose) to 181 manually paired unvaccinated ones, using predetermined matching criteria, including age, gender, and date of hospitalization. The authors reported no significant differences in mortality (19% vs. 20%, p = 0.853), even after multivariable logistic regression models (OR = 1.051, 95% CI: 0.621–1.780, p = 0.853) or when adjusted for age, gender, and number of comorbidities (aOR = 0.996, 95% CI: 0.582–1.703, p = 0.987). There were also no differences in respiratory support utilization, defined as any form of ventilatory assistance from low-flow oxygen delivery to mechanical ventilation [12].

Conversely, an American single-center cohort study observed lower in-hospital mortality rates in vaccinated patients compared to matched unvaccinated individuals, with an absolute difference of 6.5%, and an aOR of 0.57 (95% CI: 0.34–0.94; p = 0.027) [28]. This analysis included adult COVID-19 patients hospitalized due to COVID-19 or another reason from June to September 2021, involving 262 fully vaccinated and 262 matched unvaccinated ones. Propensity score matching was employed, considering factors such as age, sex, race/ethnicity, smoking, comorbidities, and BMI. However, the authors did not assess IMV in their study [28].

Another American study, with a limited sample size of 46 vaccinated patients matched with 46 unvaccinated ones, hospitalized from May 2021 to September 2021, used PS matching based on age (within 2 years) and Charlson Comorbidity Index percentage. In this study, no significant differences in mortality (13.0% vs. 15.2%, p = 0.383) or IMV were observed (19.6% vs. 15.2%, p = 0.291), even after adjusting for obesity [13].

In our secondary outcomes, vaccinated patients had an absolute difference of 11.4% in NIMV, 6.5% in dialysis, 11.5% in vasoactive drugs, 12.1% in ICU admissions, 3.8% in thrombosis and they also had a difference in medians of less 2 days in hospital length of stay. Most previously matched studies also reported lower rates of ICU admission, with an absolute difference ranging from 6.1% [28]; to 7.1% [12] and 15.2% [13]. The aforementioned matched studies have not shown differences in the other secondary outcomes between groups, or those outcomes have not been assessed [12, 13, 28]. They probably lacked the power needed to observe differences. Our study’s larger sample size and comprehensive assessment of outcomes provide a more detailed evaluation of vaccination benefits.

Interestingly, in our analysis vaccinated patients also had lower rates of thromboembolic events (7.7% vs. 3.9%, p = 0.007) including pulmonary embolism (5.9% vs. 3.4%, p = 0.047) despite similar thromboprophylaxis use. This finding aligns with a large American multicenter case-control study which reported lower rates of venous thromboembolism in Delta wave (4.4% vs.9.1, p < 0.001) [31].

Our results are also in line with a recently published large cohort study based on national vaccination campaigns using electronic health records from the UK, Spain, and Estonia. In this study, data from over 20 million patients (10.17 million vaccinated and 10.39 million unvaccinated) were assessed, and vaccination was associated with a reduced risk of venous thromboembolism and arterial thrombosis for both acute (30-day) and post-acute (31 to 365 days) COVID infection [32]. These data not only contradict claims made by anti-vaccine proponents, who argue that COVID-19 vaccines could increase the risk of thrombosis, but suggest the contrary, that vaccination against COVID-19 may confer a protective effect against thromboembolic events after infection or during COVID-19 hospitalization.

Whereas our analysis focused on hospitalized COVID-19 patients, other larger studies have used matched strategies but also included general population or ambulatory patients. A population-based Taiwanese observational study conducted during a predominant Omicron wave found reduced COVID-19 mortality and hospitalization rates among vaccinated individuals compared to unvaccinated ones. The benefits were more evident among those who had completed three vaccine doses (two primary doses and one booster) or received a protein-based vaccine as the primary one [33].

Additionally, a case-control study conducted in Hong Kong during the Omicron wave matched 1,781 cases to 1,737 controls aged 3 to 105 years based on PS. The authors found that two doses of CoronaVac were poorly protective against severe disease in individuals aged ≥ 60 years, but vaccine effectiveness increased substantially after the third dose. This study highlights the importance of booster doses and further supports the protective effect of COVID-19 vaccines observed in our study [34].

Another population-based observational study conducted in Hong Kong among hospitalized COVID-19 patients aimed to estimate the vaccine effectiveness of one, two, and three doses of both the BNT162b2 and CoronaVac vaccines. The study observed that two doses of either vaccine protected against severe disease and death within 28 days of a positive test [35].

These studies reinforce our findings that COVID-19 vaccination significantly reduces severe outcomes and mortality, highlighting the critical role of vaccines in high-risk populations.

This study has limitations. It is a retrospective analysis reliant on patient records. While we collected data on vaccine doses and brands, we did not gather detailed information on the number of doses for each brand. Although Brazil has implemented a centralized computerized vaccination system to verify individual vaccination records, the data collected for this study was de-identified to comply with ethical regulations, preventing us from checking individual vaccine information for each included patient. Therefore, while we collected data on vaccine doses and brands, we were unable to capture detailed information on the specific number of doses of each vaccine brand administered to each patient.

There was a high frequency of missing data on vaccination status, and we could not determine the vaccination date as it was not reported. Therefore, we could not affirm whether it happened 14 days before admission, which is established as the interval necessary to build an immune response after vaccination. Furthermore, for the data available, the sample size was too small to compare the effect of different vaccine brands and schedules. However, despite this limitation, we could observe undoubtedly lower rates of severe outcomes among those who were vaccinated. Additionally, prior COVID-19 infections have not been assessed, and previous natural infections have been associated with some protection against severe illness [36].

Furthermore, although our study included data from 27 hospitals in five Brazilian states, our findings may not be generalizable to the entire country, mainly because of the heterogeneity of the Brazilian population and the variation in the level of care across different regions. Virus sequencing was not carried out, making it impossible to define which variant caused hospitalization, leading to assumptions based on the predominant variant during the study period. Nevertheless, over 70% of the matched sample was captured during the second wave, which coincided with the Delta-dominant period, characterized by a highly virulent variant.

This study also has several strengths. We applied advanced analytical techniques, including PS matching, IPW, and the inclusion of PS as a covariate. For each method, we conducted logistic regression analyses, which helped to minimize potential confounding factors. Additionally, we utilized a machine learning approach to better understand the influence of vaccination and other variables on the outcomes assessed.

We included only hospitalized patients admitted with a positive test, for COVID-19 in the context of symptoms, and COVID-19 was the main reason for hospitalization. Patients admitted for other reasons with a positive test for COVID-19 were excluded. This is important because these patients, also known as “hospital-manifested COVID-19”, may have different clinical courses during hospitalization and studies have shown that they have higher mortality and ICU admission rates [37]. Our study offers a real-life perspective of variables and outcomes of the population hospitalized with COVID-19 over a 13-month observational period.

Furthermore, we evaluated all vaccines approved for use in Brazil, including multiple vaccines and mixed immunization schemes. Although we could not estimate the efficacy of each specific scheme, this can be helpful when assessing policies for future vaccine implementation, especially in countries where vaccination campaigns are still in the early stages. In our matching strategy, we used specific diseases known to impact COVID severity and not groups of disease as other authors did [12, 28], which theoretically would facilitate the matching strategy but certainly would not create real balanced groups. In contrast to other authors [12, 13], we also included obesity in the PS analysis because, in addition to its importance in COVID-19 prognosis [38], it was the only comorbidity most common in unmatched unvaccinated patients. Finally, another strength is that we conducted a notably comprehensive study, which represents one of the largest matched analyses to date, comparing various objective outcomes among hospitalized populations.

Conclusions

In conclusion, this study highlights the consistent benefits of full vaccination among patients hospitalized with COVID-19 in Brazil. Fully vaccinated patients experienced lower mortality rates, reduced severe outcomes, fewer ICU admissions, and shorter hospital stays. These underscore the critical role of vaccination in reducing COVID-19 severity and mortality. Our findings have important implications for public health strategies and support the critical importance of vaccination efforts, particularly in low-income countries, where vaccination coverage remains suboptimal, especially for completing primary series. Furthermore, our findings diminish concerns about vaccine safety, particularly regarding thrombogenesis, reinforcing the importance of widespread vaccination campaigns.

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

aOR:

Adjusted Odds ratio

CI:

Confidence intervals

COVID-19:

Coronavirus disease 2019

ICUs:

Intensive care units

IQR:

Interquartile range

IMV:

Invasive mechanical ventilation

NIMV:

Noninvasive mechanical ventilation

IPW:

Inverse probability weighting

OR:

Odds ratio

PS:

Propensity score

SARS-CoV-2:

Severe acute respiratory syndrome coronavirus 2

References

  1. Coronavirus disease (COVID-19). – World Health Organization. https://www.who.int/emergencies/diseases/novel-coronavirus-2019. Accessed 3 Aug 2024.

  2. Fürst T, Bazalová A, Fryčák T, Janošek J. Does the healthy vaccinee bias rule them all? Association of COVID-19 vaccination status and all-cause mortality from an analysis of data from 2.2 million individual health records. Int J Infect Dis. 2024;142:106976.

    Article  PubMed  Google Scholar 

  3. [No title]. https://extranet.who.int/prequal/sites/default/files/document_files/Status_COVID_VAX_08AUgust2023.pdf. Accessed 3 Aug 2024.

  4. Zheng C, Shao W, Chen X, Zhang B, Wang G, Zhang W. Real-world effectiveness of COVID-19 vaccines: a literature review and meta-analysis. Int J Infect Dis. 2022;114:252–60.

    Article  CAS  PubMed  Google Scholar 

  5. Jiesisibieke ZL, Liu W-Y, Yang Y-P, Chien C-W, Tung T-H. Effectiveness and safety of COVID-19 vaccinations: an Umbrella Meta-Analysis. Int J Public Health. 2023;68:1605526.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Ikeokwu AE, Adeniran OF, Marwizi FM, Kolade-Ernest OJ, Solomon RO, Ogedengbe W, Onyemarin-Henry P, Okpo NC, Onyinye O. A Meta-analysis to ascertain the effectiveness of COVID-19 vaccines on clinical outcomes in patients with COVID-19 infection in North America. Cureus. 2023;15:e41053.

    PubMed  PubMed Central  Google Scholar 

  7. Mohammed H, Pham-Tran DD, Yeoh ZYM, Wang B, McMillan M, Andraweera PH, Marshall HS. (2023) A systematic review and Meta-analysis on the real-world effectiveness of COVID-19 vaccines against infection, symptomatic and severe COVID-19 Disease caused by the Omicron variant (B.1.1.529). Vaccines (Basel). https://doi.org/10.3390/vaccines11020224

  8. Pratama NR, Wafa IA, Budi DS, Sutanto H, Asmarawati TP, Barlian Effendi G, Wungu CDK. (2022) Effectiveness of COVID-19 vaccines against SARS-CoV-2 Omicron variant (B.1.1.529): a systematic review with Meta-analysis and Meta-regression. Vaccines (Basel). https://doi.org/10.3390/vaccines10122180

  9. Castro MC, Kim S, Barberia L, Ribeiro AF, Gurzenda S, Ribeiro KB, Abbott E, Blossom J, Rache B, Singer BH. Spatiotemporal pattern of COVID-19 spread in Brazil. Science. 2021;372:821–6.

    Article  CAS  PubMed  Google Scholar 

  10. Vacinometro COVID-19. https://infoms.saude.gov.br/extensions/SEIDIGI_DEMAS_Vacina_C19/SEIDIGI_DEMAS_Vacina_C19.html. Accessed 3 Aug 2024.

  11. Coronavírus Brasil. https://covid.saude.gov.br/. Accessed 29 Apr 2024.

  12. Lombardi A, Villa S, Colaneri M, et al. Clinical characteristics and outcomes of vaccinated patients hospitalized with SARS-CoV-2 breakthrough infection: Multi-IPV, a multicentre study in Northern Italy. J Infect Public Health. 2024;17:467–73.

    Article  PubMed  Google Scholar 

  13. Balian S, Bailey B, Abboud S, et al. Comparative admission rates and infection severity of COVID-19 among unvaccinated and vaccinated patients. J Investig Med. 2023;71:329–38.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Paggi R, Barbiero A, Manciulli T, et al. Characteristics of COVID-19 vaccinated and unvaccinated patients admitted to Careggi University Hospital, Florence, Italy. Intern Emerg Med. 2023;18:821–30.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Marcolino MS, Pires MC, Ramos LEF, et al. ABC2-SPH risk score for in-hospital mortality in COVID-19 patients: development, external validation and comparison with other available scores. Int J Infect Dis. 2021;110:281–308.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Organization WH, Others. Diagnostic testing for SARS-CoV-2: interim guidance, 11 September 2020. World Health Organization; 2020.

  17. Moura EC, Cortez-Escalante J, Cavalcante FV, Barreto IC, de Sanchez HC, Santos MN LMP. Covid-19: temporal evolution and immunization in the three epidemiological waves, Brazil, 2020–2022. Rev Saude Publica. 2022;56:105.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inf. 2009;42:377–81.

    Article  Google Scholar 

  19. Harris PA, Taylor R, Minor BL, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inf. 2019;95:103208.

    Article  Google Scholar 

  20. Soriano Marcolino M, Minelli Figueira R, Pereira Afonso Dos Santos J, Silva Cardoso C, Luiz Ribeiro A, Alkmim MB. The experience of a sustainable large scale Brazilian Telehealth Network. Telemed J E Health. 2016;22:899–908.

    Article  PubMed  Google Scholar 

  21. Bicalho MAC, Aliberti MJR, Delfino-Pereira P, et al. Clinical characteristics and outcomes of COVID-19 patients with preexisting dementia: a large multicenter propensity-matched Brazilian cohort study. BMC Geriatr. 2024;24:25.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Marcolino MS, Ziegelmann PK, Souza-Silva MVR, et al. Clinical characteristics and outcomes of patients hospitalized with COVID-19 in Brazil: results from the Brazilian COVID-19 registry. Int J Infect Dis. 2021;107:300–10.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Breiman L. Random forests. Mach Learn. 2001;45:5–32.

    Article  Google Scholar 

  24. Molnar C. (2020) Interpretable Machine Learning. Lulu.com.

  25. XGBoost. https://doi.org/10.1145/2939672.2939785

  26. Havers FP, Pham H, Taylor CA, et al. COVID-19-Associated hospitalizations among vaccinated and unvaccinated adults 18 years or older in 13 US States, January 2021 to April 2022. JAMA Intern Med. 2022;182:1071–81.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Whittaker R, Bråthen Kristofferson A, Valcarcel Salamanca B, Seppälä E, Golestani K, Kvåle R, Watle SV, Buanes EA. Length of hospital stay and risk of intensive care admission and in-hospital death among COVID-19 patients in Norway: a register-based cohort study comparing patients fully vaccinated with an mRNA vaccine to unvaccinated patients. Clin Microbiol Infect. 2022;28:871–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Niu J, Samuels S, Sareli C, Mayer D, Visbal A, Sareli AE. Clinical features and outcomes of hospitalized adult patients with breakthrough COVID-19 infections: a propensity-score-matched observational study. Am J Epidemiol. 2024;193:285–95.

    Article  PubMed  Google Scholar 

  29. Agrawal U, Katikireddi SV, McCowan C, et al. COVID-19 hospital admissions and deaths after BNT162b2 and ChAdOx1 nCoV-19 vaccinations in 2·57 million people in Scotland (EAVE II): a prospective cohort study. Lancet Respir Med. 2021;9:1439–49.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Coronavírus. In: Ministério da Saúde. https://www.gov.br/saude/pt-br/assuntos/coronavirus/coronavirus. Accessed 8 Mar 2024.

  31. Lauring AS, Tenforde MW, Chappell JD, et al. Clinical severity of, and effectiveness of mRNA vaccines against, covid-19 from omicron, delta, and alpha SARS-CoV-2 variants in the United States: prospective observational study. BMJ. 2022;376:e069761.

    Article  PubMed  Google Scholar 

  32. Mercadé-Besora N, Li X, Kolde R, et al. The role of COVID-19 vaccines in preventing post-COVID-19 thromboembolic and cardiovascular complications. Heart. 2024. https://doi.org/10.1136/heartjnl-2023-323483.

    Article  PubMed  Google Scholar 

  33. Lee C-Y, Kuo H-W, Liu Y-L, Chuang J-H, Chou J-H. Population-based evaluation of Vaccine Effectiveness against SARS-CoV-2 infection, severe illness, and death, Taiwan. Emerg Infect Dis. 2024;30:478–89.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Ng RWY, Sze RKH, Chong KC, et al. Effectiveness of mRNA and inactivated COVID-19 vaccines: a test-negative study in an infection-naïve Hong Kong population. J Infect. 2023;87:136–43.

    Article  CAS  PubMed  Google Scholar 

  35. McMenamin ME, Nealon J, Lin Y, Wong JY, Cheung JK, Lau EHY, Wu P, Leung GM, Cowling BJ. Vaccine effectiveness of one, two, and three doses of BNT162b2 and CoronaVac against COVID-19 in Hong Kong: a population-based observational study. Lancet Infect Dis. 2022;22:1435–43.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Goldberg Y, Mandel M, Woodbridge Y, Fluss R, Novikov I, Yaari R, Ziv A, Freedman L, Huppert A. Similarity of Protection conferred by previous SARS-CoV-2 infection and by BNT162b2 vaccine: a 3-Month Nationwide Experience from Israel. Am J Epidemiol. 2022;191:1420–8.

    Article  PubMed  Google Scholar 

  37. Delfino-Pereira P, Pires MC, Gomes VMR, et al. Clinical characteristics and outcomes of hospital-manifested COVID-19 among brazilians. Int J Infect Dis. 2023;130:31–7.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Li C, Islam N, Gutierrez JP, Gutiérrez-Barreto SE, Castañeda Prado A, Moolenaar RL, Lacey B, Richter P. Associations of diabetes, hypertension and obesity with COVID-19 mortality: a systematic review and meta-analysis. BMJ Glob Health. 2023. https://doi.org/10.1136/bmjgh-2023-012581.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We would like to thank the hospitals that are part of this collaboration for supporting this project: Hospital Mater Dei - Santo Agostinho; Hospital Mater Dei - Contorno; Hospital Mater Dei - Betim-Contagem; Hospital Santo Antônio; Hospital Eduardo de Menezes; Hospital Metropolitano Dr. Célio de Castro; Hospital Metropolitano Odilon Behrens; Hospital Risoleta Tolentino Neves; Hospital Santa Rosália; Hospital Santa Cruz; Hospital São João de Deus; Hospital Semper; Hospital Universitário Canoas; Hospital Universitário Santa Maria; Hospital Nossa Senhora da Conceição; Hospital João XXIII; Hospital Júlia Kubitschek; Hospital Mãe de Deus; Hospital Regional do Oeste; Hospital das Clínicas da Faculdade de Medicina de Botucatu; Hospital Universitário Ciências Médicas; Hospital Bruno Born; Hospital Cristo Redentor; Hospital Nossa Senhora de Lourdes; Orizonti - Instituto de Saúde e Longevidade Ltda; Santa Casa de Misericórdia de Belo Horizonte; Hospital Universitário Professor Edgar Santos; Hospital Sofia Feldman. We also thank all the clinical staff at those hospitals who cared for the patients and all the undergraduate students who helped with data collection.

Funding

This study was supported in part by the Minas Gerais State Agency for Research and Development (Fundação de Amparo à Pesquisa do Estado de Minas Gerais-FAPEMIG) [grant number APQ-01154–21] and the National Institute of Science and Technology for Health Technology Assessment (Instituto de Avaliação de Tecnologias em Saúde-IATS) [grant number 465518/2014-1]. MSM was supported in part by CNPq [grant number 310561/2021–3]. The funding bodies played no role in the design of the study, the collection, analysis, and interpretation of the data, or in the writing of the manuscript.

Author information

Authors and Affiliations

  1. Medical School and University Hospital, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 110, Belo Horizonte, CEP 30130-100, Brazil

    Eduardo Villela de Moraes, Amanda Abrantes Abreu Costa, Unaí Tupinambas & Milena Soriano Marcolino

  2. Department of Statistics, Universidade Federal de Minas Gerais, Av. Presidente Antônio Carlos, 6627, ICEx, sala 4071, Belo Horizonte, Brazil

    Magda Carvalho Pires

  3. Hospital Unimed-BH, Av. do Contorno, Belo Horizonte, 3097, Brazil

    Aline Gabrielle Sousa Nunes

  4. Universidade Federal dos Vales do Jequitinhonha e Mucuri (UFVJM), R. da Glória, 187, Diamantina, Brazil

    Caroline Lopes de Amorim

  5. Hospital Universitário Canoas, Av. Farroupilha, Porto Alegre, 8001, Brazil

    Euler Roberto Fernandes Manenti & Karen Brasil Ruschel

  6. Hospital Mãe de Deus, R. José de Alencar, 286, Porto Alegre, Brazil

    Euler Roberto Fernandes Manenti & Karen Brasil Ruschel

  7. Hospital Santo Antônio, Praça Dr. Márcio Carvalho Lopes Filho, 501, Curvelo, Brazil

    Fernanda Barbosa Lucas

  8. University Hospital of Porto Alegre, R. Ramiro Barcelos, Porto Alegre, 2350, Brazil

    Fernanda d’Athayde Rodrigues

  9. Hospital Nossa Senhora da Conceição, Av. Francisco Trein, 326, Porto Alegre, Brazil

    Fernando Anschau

  10. Hospital Cristo Redentor, Rua Domingos Rubbo, 20, Porto Alegre, Brazil

    Fernando Anschau

  11. Universidade Federal de São João del-Rei, Praça Frei Orlando, 170, São João del- Rei, Brazil

    Guilherme Fonseca do Nascimento & Leonardo Chaves Dutra da Rocha

  12. Hospital SOS Cárdio, Rodovia SC-401, 121, Florianópolis, Brazil

    Giovanna Grunewald Vietta

  13. Universidade Federal da Bahia, Av. Milton Santos, Vitória da Conquista, Brazil

    Jessica Fernandes Benavides Moreira

  14. Institute for Health and Technology Assessment, R. Ramiro Barcelos, Porto Alegre, 2350, Brazil

    Karen Brasil Ruschel & Milena Soriano Marcolino

  15. Hospital Risoleta Tolentino Neves, R. das Gabirobas, 1, Belo Horizonte, Brazil

    Monica Aparecida Costa

  16. Pontifıcia Universidade Católica de Minas Gerais (PUCMG), R. Dom José Gaspar, 500, Belo Horizonte, Brazil

    Pamela Andrea Alves Duraes

  17. Centro Infant, Department of Pediatrics, School of Medicine, Pontifıcia Universidade Católica do Rio Grande do Sul (PUCRS), Av. Ipiranga, Porto Alegre, 6681, Brazil

    Pedro Augusto Van Der Sand Germani

  18. Hospital Metropolitano Doutor Célio de Castro, R. Dona Luiza, 311, Belo Horizonte, Brazil

    Priscilla Pereira dos Reis

  19. Hospital Santa Cruz, R. Fernando Abott, 174, Santa Cruz do Sul, Brazil

    Rochele Mosmann Menezes

  20. Computer science department of Universidade Federal de São João del-Rei, Praça Frei Orlando, 170, São João del-Rei, Brazil

    Leonardo Chaves Dutra da Rocha

  21. Computer Science Department of Universidade Federal de Minas Gerais, Av. Presidente Antônio Carlos, Belo Horizonte, 6627, Brazil

    Marcos André Gonçalves

  22. Department of Internal Medicine, Medical School & Telehealth Center, University Hospital, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 110, Belo Horizonte, Brazil

    Milena Soriano Marcolino

Authors
  1. Eduardo Villela de Moraes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Magda Carvalho Pires
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amanda Abrantes Abreu Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aline Gabrielle Sousa Nunes
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Caroline Lopes de Amorim
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Euler Roberto Fernandes Manenti
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Fernanda Barbosa Lucas
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Fernanda d’Athayde Rodrigues
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Fernando Anschau
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Guilherme Fonseca do Nascimento
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Giovanna Grunewald Vietta
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Jessica Fernandes Benavides Moreira
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Karen Brasil Ruschel
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Monica Aparecida Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Pamela Andrea Alves Duraes
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Pedro Augusto Van Der Sand Germani
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Priscilla Pereira dos Reis
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Rochele Mosmann Menezes
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Leonardo Chaves Dutra da Rocha
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Marcos André Gonçalves
    View author publications

    You can also search for this author in PubMed Google Scholar

  21. Unaí Tupinambas
    View author publications

    You can also search for this author in PubMed Google Scholar

  22. Milena Soriano Marcolino
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Substantial contributions to the conception or design of the manuscript: Moraes EV, Pires MC, Tupinambás U and Marcolino MS. Substantial contributions for data acquisition, analysis, or interpretation: Moraes EV, Pires MC, Costa AAA, Nunes AGS, Amorim CL, Manenti ERF, Lucas FB, D’Athayde Rodrigues F, Anschau F, Vietta GG, Moreira JFB, Ruschel KB, Costa MA, Duraes PAA, Germani PAVDS, Reis PP, Menezes RM, Nascimento GF, Rocha LCD, Gonçalves MA, Tupinambás U and Marcolino MS. Manuscript formulation: Moraes EV, Pires MC, Tupinambás U and Marcolino MS. We agree to be responsible for all aspects of the work, ensuring that issues related to the precision of integrity in any of the work’s parts will be properly investigated and solved: all authors. Revised the manuscript critically for important intellectual content: all authors. Final approval of the version to be published: all authors.

Corresponding author

Correspondence to Eduardo Villela de Moraes.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the National Research Ethics Committee (CAAE: 30350820.5.1001.0008). The study adhered to the Declaration of Helsinki. Individual informed consent was waived due to the seriousness of the situation imposed by the pandemic and the retrospective nature of the study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary Material 2

Supplementary Material 3

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Moraes, E.V., Pires, M.C., Costa, A.A.A. et al. Comprehensive statistical analysis reveals significant benefits of COVID-19 vaccination in hospitalized patients: propensity score, covariate adjustment, and feature importance by permutation. BMC Infect Dis 24, 1052 (2024). https://doi.org/10.1186/s12879-024-09865-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12879-024-09865-1

Keywords

BMC Infectious Diseases

ISSN: 1471-2334

Contact us