Effectiveness of oseltamivir treatment on clinical failure in hospitalized patients with lower respiratory tract infection

Abstract

Background

Influenza is associated with excess morbidity and mortality of individuals each year. Few therapies exist for treatment of influenza infection, and each require initiation as early as possible in the course of infection, making efficacy difficult to estimate in the hospitalized patient with lower respiratory tract infection. Using causal machine learning methods, we re-analyze data from a randomized trial of oseltamivir versus standard of care aimed at reducing clinical failure in hospitalized patients with lower respiratory tract infection during the influenza season.

Methods

This was a secondary analysis of the Rapid Empiric Treatment with Oseltamivir Study (RETOS). Conditional average treatment effects (CATE) and 95% confidence intervals were computed from causal forest including 85 clinical and demographic variables. RETOS was a multicenter, randomized, unblinded, trial of adult patients hospitalized with lower respiratory tract infections in Kentucky from 2009 through 2012. Adult hospitalized patients with lower respiratory tract infection were randomized to standard of care or standard of care plus oseltamivir as early as possible after hospital admission but within 24 h of enrollment. After randomization, oseltamivir was initiated in the treatment arm per package insert. The primary outcome was clinical failure, a composite measure including failure to reach clinical improvement within 7 days, transfer to intensive care 24 h after admission, or rehospitalization or death within 30 days.

Results

A total of 691 hospitalized patients with lower respiratory tract infections were included in the study. The only subgroup of patients with a statistically significant CATE was those with laboratory-confirmed influenza infection with a 26% lower risk of clinical failure when treated with oseltamivir (95% CI 3.2–48.0%).

Conclusions

This study suggests that addition of oseltamivir to standard of care may decrease clinical failure in hospitalized patients with influenza-associated lower respiratory tract infection versus standard of care alone. These results are supportive of current recommendations to initiate antiviral treatment in hospitalized patients with confirmed or suspected influenza as soon as possible after admission.

Trial registration Original trial: Clinical Trials.Gov; Rapid Empiric Treatment With Oseltamivir Study (RETOS) (RETOS); ClinicalTrials.gov Identifier: NCT01248715 https://clinicaltrials.gov/ct2/show/NCT01248715

Introduction

Influenza is associated with excess morbidity and mortality of individuals each year in the United States and contributes substantially to the national healthcare burden each winter [1]. Neuraminidase inhibitors such as oseltamivir, peramivir, and zanamivir are one of three categories of FDA-approved therapies for influenza illness and reduce duration of infection through prevention of virus exit from infected cells [2]. Because of this mechanism, administration of drug early in the course of infection is most efficacious [3]. We previously conducted a randomized trial to evaluate the impact of oseltamivir on clinical outcomes of hospitalized patients with lower respiratory tract infection associated with influenza [4]. In that study, we found limited, and not statistically significant, efficacy of oseltamivir in reducing clinical failure, a composite measure including failure to reach clinical improvement within 7 days, transfer to intensive care 24 h after admission, or rehospitalization or death within 30 days, in hospitalized patients with influenza-associated lower respiratory tract infection [4]. Although the average treatment effect was not significant, it is possible that therapy had clinical benefits in subgroups of patients, or our analytical approach was insufficient for the data obtained.

Since the results of this study were published, there have been many innovations in analytical approaches for these types of data, specifically the field of machine learning [5]. These advancements have improved not only our ability to develop predictive models but also allow for computation of treatment effects. In this area, treatment effect computation is also possible across subgroups of individuals, with fewer limitations of sample size and false discovery rates that plague frequentist statistical approaches [6]. Since the sample size of influenza virus infected patients in our initial randomized trial was relatively small and we were underpowered for our primary endpoint, it is possible that we were unable to appropriately detect subgroups in which oseltamivir therapy was efficacious.

The objective of this post hoc study was to utilize a novel machine learning method, the causal forest [7], to evaluate subgroups of hospitalized patients with lower respiratory tract infection who may have differential therapeutic response to oseltamivir therapy for prevention of clinical failure.

Methods

Design and patients

This was a secondary analysis of the Rapid Empiric Treatment with Oseltamivir Study (RETOS) [4]. Briefly, RETOS was a randomized, unblinded, trial of adult patients hospitalized with lower respiratory tract infections in Kentucky from 2009 through 2012. Patients were randomized to group A (standard of care) or group B (standard of care plus oseltamivir) as early as possible after hospital admission but within 24 h of enrollment. Both per-protocol and intent-to-treat analyses were performed in the original study since all patients with lower respiratory tract infections were randomized regardless of etiology, though subsets with documented influenza virus infection by reverse transcriptase polymerase chain reaction (rt-PCR) were also analyzed. For the purposes of the present study, all patients in the intent-to-treat analysis (randomized patients with lower respiratory tract infection regardless of documented etiology) were included.

Study variables

A total of 85 variables were used to investigate potential heterogeneity in average treatment effects between oseltamivir and clinical failure. All variables used in the models are included in Fig. 1 of the results. Variables were selected based on clinical interest, complete data availability in the study database, and potential need for adjustment due to confounding effects in the assessment of treatment effects of oseltamivir on clinical failure.

Fig. 1

Conditional average treatment effects for clinical failure. Variable listed is the subgroup of patients for which the treatment effect was computed for patients treated versus not treated with oseltamivir

Study outcomes

The primary outcome was clinical failure, as defined in our original study: a composite measure including failure to reach clinical improvement within 7 days, transfer to intensive care 24 h after admission, or rehospitalization or death within 30 days.

Human subjects protection

The University of Louisville Human Subjects Research Protection Program Office (Protocol 10.0465), the Robley Rex VA Medical Center Institutional Review Board (IRB; Protocol 0068/00325), and each participating hospital reviewed and approved the study prior to any enrollment. The Centers for Disease Control and Prevention (CDC) IRB granted reliance on local ethical review approvals as the funding agency of the original study.

Statistical analysis

Baseline characteristics of patients were compared using Chi-Squared or Fishers Exact tests. Q values were also computed for all variables versus treatment status to account for the increased false discovery rate due to multiple comparison in this analysis. Causal forests were used to estimate conditional average treatment effects for each variable for those treated and untreated with oseltamivir [6,7,8,9]. These heterogenous effects were also accounting for all other variables under study via the random forest approach and can be considered unbiased estimates of the absolute effect of the oseltamivir therapy conditioned on membership in a particular subgroup. Causal forests are extensions of the random forest, which split data repeatedly to create decision trees for classification or regression. The causal forest is built similarly, though instead of assessing the variable selection (splitting rules) based on prediction error, it maximizes the difference in treatment effects for the treatment/outcome pair. For the causal forest, a total of 50,000 trees were used to allow for accurate computation of 95% confidence intervals. Conditional average treatment effects were extracted for each categorical variable under consideration along with the 95% confidence interval and a data visualization was created for each outcome. R v4.04 (R Foundation for Statistical Computing, Vienna, Austria) was used for all analyses. The package grf was used for computation of causal forests and extraction of conditional average treatment effects [10].

Results

A total of 691 hospitalized patients with lower respiratory tract infections randomized to either standard of care or standard of care plus oseltamivir with complete data for the variables under consideration were included in the study. In bivariable analyses (Table 1), two variables (Age category ≥ 75 years and Wednesday admission to the hospital) were significantly different across treatment status levels based on p-values, though q-values for all variables were 1, indicating statistically significant p-values were likely due to false discovery as opposed to true differences. Figure 1 depicts the conditional average treatment effect estimates for each variable under study for those treated with oseltamivir versus those who were not treated. For the primary outcome of clinical failure, patients who were documented influenza positive were the only subgroup identified with significant oseltamivir impact. The results identified a 26% lower occurrence of clinical failure for patients with influenza treated with oseltamivir versus those not treated with oseltamivir (95% CI 3.2–48.0%).

Table 1 Bivariable analysis comparing each of the selected subgroups to study arm

Discussion

This study suggests that addition of oseltamivir to standard of care may decrease clinical failure in hospitalized patients with influenza-associated lower respiratory tract infection versus standard of care alone. These results are reasonably consistent with the reductions in clinical failure with oseltamivir treatment in the per-protocol analysis from our original randomized trial which identified a non-significant reduction from 24% in the standard of care arm to 14% in the standard of care plus oseltamivir arm (p = 0.414) [4].

Causal inference in machine learning is a relatively new addition to the methodologic arsenal [5]. The causal forest, a special version of the generalized random forest [7], is the most widely used method for computing conditional average treatment effects in the medical literature; being used for readmission risk [11], targeted intervention development [6], diabetes epidemiology [12], and identification of risk factors for thyroid disease [13]. As discussed previously, causal machine learning methods are distinct from traditional supervised machine learning since these models estimate the treatment effect as opposed to risk or prediction. Further, they are not bound by parametric assumptions common in traditional methods such as regression modeling and are less apt to overfit through application of regularization in the computation [8, 11]. These factors, along with the ability to assess conditional average treatment effects in small sample sizes allowed us to perform a more robust analysis of these randomized data.

Here, the presence of influenza was the primary driver of significant decreases in clinical failure with oseltamivir therapy. This suggests that our approach to estimate conditional average treatment effects through causal machine learning methods is likely accurate and potentially more useful for detection of subgroup treatment effects in small samples. Future randomized trials may benefit from using similar methodologies as an adjunctive measure for the more traditional frequentist statistical methodologies typically utilized and reported. Use of these novel methods may assist in detection of subgroups where interventions are beneficial or detrimental, allowing for a movement toward more personalized medicine.

This study has several limitations. First, given the small sample size in the study, the variability in our treatment effect estimates is wide, as indicated by many of the 95% confidence intervals for many variables. Further, we were not able to assess model performance through training and testing given the small sample size, resulting in potentially biased results. Second, the generalizability of treatment effect estimates from causal forest methodologies has yet to be widely documented. These machine learning models for computation of heterogenous treatment effects have only begun to be utilized in medicine [6, 13, 14] and we were unable to find any studies using these methods in the field of respiratory infections. This study also used a composite outcome, combining several clinical outcomes: failure to reach clinical improvement within 7 days, transfer to intensive care 24 h after admission, or rehospitalization or death within 30 days. Because of this, we are not able to dissect which individual outcome is impacted by oseltamivir therapy. Further, given this is a novel computational approach for re-analysis of a single randomized clinical trial, we are unable to provide further clinical guidance on use of oseltamivir in the hospitalized patient and suggest continued application of national, regional, and local guidance on anti-influenza therapy.

The strengths of this study include both the data used from the largest randomized trial of hospitalized patients with oseltamivir therapy and the consistency of results from the initial trial using traditional methodologies such as regression modeling.

Future studies may benefit from these methods as adjunctive analytics in randomized trials and potentially for observational designs where appropriate adjustments can be made with collected data. By continuing to perform both methods, we can begin to identify the best analytic approaches to identify more targeted treatments to improve patient outcomes.

In conclusion, this secondary analysis of a randomized clinical trial suggests that oseltamivir may have clinical utility in hospitalized patients with influenza-associated lower respiratory tract infections. These results are supportive of current recommendations to initiate antiviral treatment in hospitalized patients with confirmed or suspected influenza as soon as possible after admission [3].