- Open Access
Effectiveness of oseltamivir treatment on clinical failure in hospitalized patients with lower respiratory tract infection
BMC Infectious Diseases volume 21, Article number: 1106 (2021)
Influenza is associated with excess morbidity and mortality of individuals each year. Few therapies exist for treatment of influenza infection, and each require initiation as early as possible in the course of infection, making efficacy difficult to estimate in the hospitalized patient with lower respiratory tract infection. Using causal machine learning methods, we re-analyze data from a randomized trial of oseltamivir versus standard of care aimed at reducing clinical failure in hospitalized patients with lower respiratory tract infection during the influenza season.
This was a secondary analysis of the Rapid Empiric Treatment with Oseltamivir Study (RETOS). Conditional average treatment effects (CATE) and 95% confidence intervals were computed from causal forest including 85 clinical and demographic variables. RETOS was a multicenter, randomized, unblinded, trial of adult patients hospitalized with lower respiratory tract infections in Kentucky from 2009 through 2012. Adult hospitalized patients with lower respiratory tract infection were randomized to standard of care or standard of care plus oseltamivir as early as possible after hospital admission but within 24 h of enrollment. After randomization, oseltamivir was initiated in the treatment arm per package insert. The primary outcome was clinical failure, a composite measure including failure to reach clinical improvement within 7 days, transfer to intensive care 24 h after admission, or rehospitalization or death within 30 days.
A total of 691 hospitalized patients with lower respiratory tract infections were included in the study. The only subgroup of patients with a statistically significant CATE was those with laboratory-confirmed influenza infection with a 26% lower risk of clinical failure when treated with oseltamivir (95% CI 3.2–48.0%).
This study suggests that addition of oseltamivir to standard of care may decrease clinical failure in hospitalized patients with influenza-associated lower respiratory tract infection versus standard of care alone. These results are supportive of current recommendations to initiate antiviral treatment in hospitalized patients with confirmed or suspected influenza as soon as possible after admission.
Trial registration Original trial: Clinical Trials.Gov; Rapid Empiric Treatment With Oseltamivir Study (RETOS) (RETOS); ClinicalTrials.gov Identifier: NCT01248715 https://clinicaltrials.gov/ct2/show/NCT01248715
Influenza is associated with excess morbidity and mortality of individuals each year in the United States and contributes substantially to the national healthcare burden each winter . Neuraminidase inhibitors such as oseltamivir, peramivir, and zanamivir are one of three categories of FDA-approved therapies for influenza illness and reduce duration of infection through prevention of virus exit from infected cells . Because of this mechanism, administration of drug early in the course of infection is most efficacious . We previously conducted a randomized trial to evaluate the impact of oseltamivir on clinical outcomes of hospitalized patients with lower respiratory tract infection associated with influenza . In that study, we found limited, and not statistically significant, efficacy of oseltamivir in reducing clinical failure, a composite measure including failure to reach clinical improvement within 7 days, transfer to intensive care 24 h after admission, or rehospitalization or death within 30 days, in hospitalized patients with influenza-associated lower respiratory tract infection . Although the average treatment effect was not significant, it is possible that therapy had clinical benefits in subgroups of patients, or our analytical approach was insufficient for the data obtained.
Since the results of this study were published, there have been many innovations in analytical approaches for these types of data, specifically the field of machine learning . These advancements have improved not only our ability to develop predictive models but also allow for computation of treatment effects. In this area, treatment effect computation is also possible across subgroups of individuals, with fewer limitations of sample size and false discovery rates that plague frequentist statistical approaches . Since the sample size of influenza virus infected patients in our initial randomized trial was relatively small and we were underpowered for our primary endpoint, it is possible that we were unable to appropriately detect subgroups in which oseltamivir therapy was efficacious.
The objective of this post hoc study was to utilize a novel machine learning method, the causal forest , to evaluate subgroups of hospitalized patients with lower respiratory tract infection who may have differential therapeutic response to oseltamivir therapy for prevention of clinical failure.
Design and patients
This was a secondary analysis of the Rapid Empiric Treatment with Oseltamivir Study (RETOS) . Briefly, RETOS was a randomized, unblinded, trial of adult patients hospitalized with lower respiratory tract infections in Kentucky from 2009 through 2012. Patients were randomized to group A (standard of care) or group B (standard of care plus oseltamivir) as early as possible after hospital admission but within 24 h of enrollment. Both per-protocol and intent-to-treat analyses were performed in the original study since all patients with lower respiratory tract infections were randomized regardless of etiology, though subsets with documented influenza virus infection by reverse transcriptase polymerase chain reaction (rt-PCR) were also analyzed. For the purposes of the present study, all patients in the intent-to-treat analysis (randomized patients with lower respiratory tract infection regardless of documented etiology) were included.
A total of 85 variables were used to investigate potential heterogeneity in average treatment effects between oseltamivir and clinical failure. All variables used in the models are included in Fig. 1 of the results. Variables were selected based on clinical interest, complete data availability in the study database, and potential need for adjustment due to confounding effects in the assessment of treatment effects of oseltamivir on clinical failure.
The primary outcome was clinical failure, as defined in our original study: a composite measure including failure to reach clinical improvement within 7 days, transfer to intensive care 24 h after admission, or rehospitalization or death within 30 days.
Human subjects protection
The University of Louisville Human Subjects Research Protection Program Office (Protocol 10.0465), the Robley Rex VA Medical Center Institutional Review Board (IRB; Protocol 0068/00325), and each participating hospital reviewed and approved the study prior to any enrollment. The Centers for Disease Control and Prevention (CDC) IRB granted reliance on local ethical review approvals as the funding agency of the original study.
Baseline characteristics of patients were compared using Chi-Squared or Fishers Exact tests. Q values were also computed for all variables versus treatment status to account for the increased false discovery rate due to multiple comparison in this analysis. Causal forests were used to estimate conditional average treatment effects for each variable for those treated and untreated with oseltamivir [6,7,8,9]. These heterogenous effects were also accounting for all other variables under study via the random forest approach and can be considered unbiased estimates of the absolute effect of the oseltamivir therapy conditioned on membership in a particular subgroup. Causal forests are extensions of the random forest, which split data repeatedly to create decision trees for classification or regression. The causal forest is built similarly, though instead of assessing the variable selection (splitting rules) based on prediction error, it maximizes the difference in treatment effects for the treatment/outcome pair. For the causal forest, a total of 50,000 trees were used to allow for accurate computation of 95% confidence intervals. Conditional average treatment effects were extracted for each categorical variable under consideration along with the 95% confidence interval and a data visualization was created for each outcome. R v4.04 (R Foundation for Statistical Computing, Vienna, Austria) was used for all analyses. The package grf was used for computation of causal forests and extraction of conditional average treatment effects .
A total of 691 hospitalized patients with lower respiratory tract infections randomized to either standard of care or standard of care plus oseltamivir with complete data for the variables under consideration were included in the study. In bivariable analyses (Table 1), two variables (Age category ≥ 75 years and Wednesday admission to the hospital) were significantly different across treatment status levels based on p-values, though q-values for all variables were 1, indicating statistically significant p-values were likely due to false discovery as opposed to true differences. Figure 1 depicts the conditional average treatment effect estimates for each variable under study for those treated with oseltamivir versus those who were not treated. For the primary outcome of clinical failure, patients who were documented influenza positive were the only subgroup identified with significant oseltamivir impact. The results identified a 26% lower occurrence of clinical failure for patients with influenza treated with oseltamivir versus those not treated with oseltamivir (95% CI 3.2–48.0%).
This study suggests that addition of oseltamivir to standard of care may decrease clinical failure in hospitalized patients with influenza-associated lower respiratory tract infection versus standard of care alone. These results are reasonably consistent with the reductions in clinical failure with oseltamivir treatment in the per-protocol analysis from our original randomized trial which identified a non-significant reduction from 24% in the standard of care arm to 14% in the standard of care plus oseltamivir arm (p = 0.414) .
Causal inference in machine learning is a relatively new addition to the methodologic arsenal . The causal forest, a special version of the generalized random forest , is the most widely used method for computing conditional average treatment effects in the medical literature; being used for readmission risk , targeted intervention development , diabetes epidemiology , and identification of risk factors for thyroid disease . As discussed previously, causal machine learning methods are distinct from traditional supervised machine learning since these models estimate the treatment effect as opposed to risk or prediction. Further, they are not bound by parametric assumptions common in traditional methods such as regression modeling and are less apt to overfit through application of regularization in the computation [8, 11]. These factors, along with the ability to assess conditional average treatment effects in small sample sizes allowed us to perform a more robust analysis of these randomized data.
Here, the presence of influenza was the primary driver of significant decreases in clinical failure with oseltamivir therapy. This suggests that our approach to estimate conditional average treatment effects through causal machine learning methods is likely accurate and potentially more useful for detection of subgroup treatment effects in small samples. Future randomized trials may benefit from using similar methodologies as an adjunctive measure for the more traditional frequentist statistical methodologies typically utilized and reported. Use of these novel methods may assist in detection of subgroups where interventions are beneficial or detrimental, allowing for a movement toward more personalized medicine.
This study has several limitations. First, given the small sample size in the study, the variability in our treatment effect estimates is wide, as indicated by many of the 95% confidence intervals for many variables. Further, we were not able to assess model performance through training and testing given the small sample size, resulting in potentially biased results. Second, the generalizability of treatment effect estimates from causal forest methodologies has yet to be widely documented. These machine learning models for computation of heterogenous treatment effects have only begun to be utilized in medicine [6, 13, 14] and we were unable to find any studies using these methods in the field of respiratory infections. This study also used a composite outcome, combining several clinical outcomes: failure to reach clinical improvement within 7 days, transfer to intensive care 24 h after admission, or rehospitalization or death within 30 days. Because of this, we are not able to dissect which individual outcome is impacted by oseltamivir therapy. Further, given this is a novel computational approach for re-analysis of a single randomized clinical trial, we are unable to provide further clinical guidance on use of oseltamivir in the hospitalized patient and suggest continued application of national, regional, and local guidance on anti-influenza therapy.
The strengths of this study include both the data used from the largest randomized trial of hospitalized patients with oseltamivir therapy and the consistency of results from the initial trial using traditional methodologies such as regression modeling.
Future studies may benefit from these methods as adjunctive analytics in randomized trials and potentially for observational designs where appropriate adjustments can be made with collected data. By continuing to perform both methods, we can begin to identify the best analytic approaches to identify more targeted treatments to improve patient outcomes.
In conclusion, this secondary analysis of a randomized clinical trial suggests that oseltamivir may have clinical utility in hospitalized patients with influenza-associated lower respiratory tract infections. These results are supportive of current recommendations to initiate antiviral treatment in hospitalized patients with confirmed or suspected influenza as soon as possible after admission .
Availability of data and materials
The datasets generated and/or analyzed during the current study are not publicly available as they contain protected health information but may be available in de-identified format from the corresponding author on reasonable request.
Centers for Disease Control and Prevention. Disease burden of influenza. https://www.cdc.gov/flu/about/burden/index.html. Updated 5 Oct 2020. Accessed 15 Mar 2021.
Jackson RJ, Cooper KL, Tappenden P, et al. Oseltamivir, zanamivir and amantadine in the prevention of influenza: a systematic review. J Infect. 2011;62(1):14–25.
Uyeki TM, Bernstein HH, Bradley JS, et al. Clinical Practice Guidelines by the Infectious Diseases Society of America: 2018 update on diagnosis, treatment, chemoprophylaxis, and institutional outbreak management of seasonal influenzaa. Clin Infect Dis. 2019;68(6):895–902.
Ramirez J, Peyrani P, Wiemken T, Chaves SS, Fry AM. A randomized study evaluating the effectiveness of oseltamivir initiated at the time of hospital admission in adults hospitalized with influenza-associated lower respiratory tract infections. Clin Infect Dis. 2018;67(5):736–42.
Wiemken TL, Kelley RR. Machine learning in epidemiology and health outcomes research. Annu Rev Public Health. 2020;41:21–36.
Baum A, Scarpa J, Bruzelius E, Tamler R, Basu S, Faghmous J. Targeting weight loss interventions to reduce cardiovascular complications of type 2 diabetes: a machine learning-based post-hoc analysis of heterogeneous treatment effects in the Look AHEAD trial. Lancet Diabetes Endocrinol. 2017;5(10):808–15.
Athey S, Tibshirani J, Wager S. Generalized random forests. Ann Stat. 2019;47(2):1148–78.
Wager S, Athey S. Estimation and inference of heterogeneous treatment effects using random forests. arXiv:151004342 [math, stat]. 2015.
Künzel SR, Sekhon JS, Bickel PJ, Yu B. Metalearners for estimating heterogeneous treatment effects using machine learning. Proc Natl Acad Sci. 2019;116(10):4156–65.
Generalized Random Forests [computer program]. Version 1.2.0. 2020.
Marafino BJ, Schuler A, Liu VX, Escobar GJ, Baiocchi M. Predicting preventable hospital readmissions with causal machine learning. Health Serv Res. 2020;55(6):993–1002.
Elek P, Biro A. Regional differences in diabetes across Europe—regression and causal forest analyses. Econ Hum Biol. 2021;40:100948.
Wang L, Cao F, Wang S, Sun M, Dong L. Using k-dependence causal forest to mine the most significant dependency relationships among clinical variables for thyroid disease diagnosis. PLoS ONE. 2017;12(8):e0182070.
Scarpa J, Bruzelius E, Doupe P, Le M, Faghmous J, Baum A. Assessment of risk of harm associated with intensive blood pressure management among patients with hypertension who smoke: a secondary analysis of the systolic blood pressure intervention trial. JAMA Netw Open. 2019;2(3):e190005.
This study was not funded. However, the original randomized trial was supported by an award to the University of Louisville by the CDC (Cooperative agreement IP000420-01).
Ethics approval and consent to participate
The University of Louisville Human Subjects Research Protection Program Office (Protocol 10.0465), the Robley Rex VA Medical Center Institutional Review Board (IRB; Protocol 0068/00325), and each participating hospital reviewed and approved the study prior to any enrollment. The Centers for Disease Control and Prevention (CDC) IRB granted reliance on local ethical review approvals as the funding agency of the original study. All methods were carried out in accordance with the relevant guidelines and regulations (e.g. Helsinki Declaration).
Consent for publication
The authors declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Wiemken, T.L., Furmanek, S.P., Carrico, R.M. et al. Effectiveness of oseltamivir treatment on clinical failure in hospitalized patients with lower respiratory tract infection. BMC Infect Dis 21, 1106 (2021). https://doi.org/10.1186/s12879-021-06812-2
- Heterogenous treatment effects
- Causal forest