Skip to main content
Download PDF

Comparative effectiveness of tocilizumab vs standard care in patients with severe COVID-19-related pneumonia: a retrospective cohort study utilizing registry data as a synthetic control

BMC Infectious Diseases volume 23, Article number: 849 (2023) Cite this article

Abstract

Background

The severity of coronavirus disease 2019 (COVID-19) infections has led to the development of several therapeutic agents, with tocilizumab becoming increasingly used to treat patients with COVID-19-related pneumonia. This study compared the use of tocilizumab treatment with the standard of care (SOC) to determine its efficacy against severe COVID-19-related pneumonia in Japan.

Methods

This retrospective cohort study was designed to evaluate the efficacy of tocilizumab in two different databases: the JA42434 single-arm study and COVID-19 Registry Japan (COVIREGI-JP), with a synthetic control group from the COVIREGI-JP cohort as a benchmark for the tocilizumab group. The study’s primary objective was to evaluate the efficacy of tocilizumab in treating severe COVID-19-related pneumonia compared to the SOC among patients included in the above two databases. The SOC group was extracted as the synthetic control group using exact matching and a propensity score matching in sequence per subject. As a secondary objective, the efficacy of tocilizumab compared to the SOC was evaluated exclusively among patients included in the COVIREGI-JP database. In each objective, the primary endpoint was defined as the time to discharge or the status of awaiting discharge.

Results

For the primary endpoint, the hazard ratio (HR) of the tocilizumab group against the SOC group was 1.070 (95% confidence interval [CI]: 0.565–2.028). The median time from Study Day 1 to discharge or the state of awaiting discharge was 15 days in the tocilizumab group and 16 days in the SOC group. The HRs for the secondary endpoints, namely, time to improvement in the clinical state, time to clinical failure, and time to recovery, were 1.112 (95% CI: 0.596–2.075), 0.628 (95% CI: 0.202–1.953), and 1.019 (95% CI: 0.555–1.871), respectively. Similarly, the HR of the primary endpoint for the secondary objective was 0.846 (95% CI: 0.582–1.230).

Conclusions

Tocilizumab did not demonstrate a positive effect on time to discharge or the state of awaiting discharge. Furthermore, no statistically significant differences in other clinical outcomes, such as time to improvement in the clinical state, time to clinical failure, and time to recovery, were observed among the groups.

Peer Review reports

Background

Many individuals suffering from coronavirus disease 2019 (COVID-19) infections require hospital care due to a respiratory illness. Moreover, progressive critical illness with hypoxic respiratory failure may require prolonged ventilatory support. Therapeutic agents are being developed rapidly in response to the global COVID-19 pandemic. Several studies have reported an association between interleukin (IL)-6 and COVID-19 severity, which implies that IL-6 likely is related with the cytokine storm or acute respiratory distress syndrome [1,2,3,4,5]. Tocilizumab, a recombinant humanized monoclonal antibody directed at both the soluble and membrane-bound IL-6 receptors, has been used as a therapeutic agent to treat severe COVID-19-related pneumonia [6, 7].

There have been several clinical trials evaluating the effects of tocilizumab. The RECOVERY study included 4,116 hospitalized adult patients with COVID-19 characterized by hypoxemia and systemic inflammation. This trial compared the efficacy and safety of tocilizumab and the standard of care (SOC). For patients hospitalized with severe COVID-19, treatment with tocilizumab reduced mortality, increased the probability of successful hospital discharge, and reduced the need for invasive mechanical ventilation [8]. Moreover, recent retrospective studies have suggested that tocilizumab may be associated with a lower risk of death or intubation in patients with severe COVID-19 pneumonia [9,10,11].

Owing to national differences in factors such as the number of patients with COVID-19 relative to the population and healthcare systems, there may exist national differences in COVID-19 outcomes, including mortality rates [12]. Indeed, the characteristics and outcomes of hospitalized patients in Japan differed from those reported in other countries [13, 14]. Accordingly, a phase-III clinical study of tocilizumab for treating severe COVID-19-related pneumonia (JA42434 study) was performed in Japan. This study aimed to evaluate the efficacy, safety, pharmacodynamics, and pharmacokinetics of tocilizumab administered concomitantly with the SOC. However, the number of patients infected with COVID-19 in Japan was limited at the intended commencement of the study, making it potentially challenging to conduct a randomized clinical trial. Therefore, it was planned as a multi-site single-arm study. Thus, a more detailed study evaluating the efficacy of tocilizumab was considered appropriate.

Accordingly, we conducted a retrospective cohort study to evaluate the efficacy of tocilizumab compared with the SOC using that from two different databases: the JA42434 single-arm study and COVID-19 Registry Japan (COVIREGI-JP), with a synthetic control group from the COVIREGI-JP cohort included as a benchmark for the tocilizumab group. COVIREGI-JP is the largest registry of COVID-19 hospitalized patients in Japan. It is a database of reported COVID-19 cases centered on the National Center for Global Health and Medicine [13, 14]. The patients registered have been diagnosed as having COVID-19 and hospitalized at medical institutions in Japan. As of October 24, 2022, 73,898 patients from 687 participating institutions have been registered in this database.

This study’s primary objective was to evaluate the efficacy of tocilizumab on SOC in the JA42434 population by comparing it to a synthetic control group from the SOC population of the COVIREGI-JP cohort with severe COVID-19 pneumonia. As a secondary objective, the efficacy of tocilizumab compared to SOC was evaluated exclusively among participants included in the COVIREGI-JP database. To overcome potential bias associated with utilizing a synthetic control, appropriate design and analytical methods were used, including an exact and a propensity score matching in sequence per subject [15].

Methods

Study design and approval

To evaluate the efficacy of tocilizumab and SOC treatments against severe COVID-19-related pneumonia, a retrospective cohort study was conducted using data from the JA42434 study and the COVIREGI-JP database. The definitions of each cohort and group are as follows:

  • JA42434 study tocilizumab set: patients who were registered in the JA42434 study and to whom tocilizumab was administered.

  • COVIREGI-JP cohort tocilizumab and SOC sets: among patients registered in COVIREGI-JP, the set of patients who received tocilizumab treatment (tocilizumab set), and those who did not (SOC set). We used data from patients who had all the major observation items as of November 28, 2020 (i.e., frozen data as of November 28, 2020).

  • Tocilizumab group and SOC group: matched pairs extracted from the tocilizumab and SOC sets following the matching algorithm defined beforehand. The matching was conducted separately for primary and secondary objectives.

For the primary objective, primary and secondary endpoints were compared between the tocilizumab group extracted from the JA42434 study and the SOC group extracted from the COVIREGI-JP cohort SOC set. As a secondary objective, the endpoints were compared between the tocilizumab and SOC groups, each extracted from the COVIREGI-JP cohort tocilizumab and SOC sets (Fig. 1).

Fig. 1
figure 1

Framework for the primary and secondary objectives of this study. Primary objective: Compare the outcomes of the tocilizumab group from the JA42434 single-arm study with a synthetic control group from the COVIREGI-JP cohort. Secondary objective: Compare the outcomes of the tocilizumab and control groups exclusively among participants included in COVIREGI-JP

Full size image

Ethical approval

This study was conducted in accordance with ethical principles based on the Declaration of Helsinki and in compliance with the ethical guidelines for medical research involving human participants. It was approved by the National Center for Global Health and Medicine Ethics Review Committee (NCGM-G-004119-00) and registered in April 2020 in the UMIN Clinical Trials Registry (UMIN-CTR; ID: UMIN000043622). Informed consent from each patient was obtained in the form of opt-out and information regarding the opt-out approach of our study is available on the registry website. However, the participants of the source study, JA42434 study, provided informed consent.

Study participants

The main inclusion criteria for the JA42434 study were as follows: patients who agreed to participate by signing a consent form, patients who were 18 years of age or older, patients hospitalized with COVID-19 pneumonia confirmed by a positive PCR test (respiratory, blood, urine, stool, or other bodily fluid samples) and chest X-radiography (X-ray) or computed tomography (CT) scan. In addition, the following pulmonary criteria were required: decreased oxygen saturation (SpO2 ≤ 93%) or hypoxemia (PaO2/FiO2 < 300 mmHg) under the SOC; otherwise, SpO2 ≤ 93% documented under lower supplemental oxygen or ambient air during the screening period, even if SpO2 > 93% under oxygen administration. Patients were excluded from the JA42434 study if they had a history of severe allergic reaction to tocilizumab or other monoclonal antibodies, were likely to die within 24 h regardless of treatment (in the opinion of the investigator or sub-investigator), were pregnant or lactating, or showed a positive pregnancy test before administration. In the JA42434 study, patients were enrolled from 12 enrollment sites, and the first-patient-in and last-patient-in dates were May 25, 2020 and October 31, 2020, respectively.

For the COVIREGI-JP cohort tocilizumab and SOC sets, we included patients who were at least 18 years old at the onset of COVID-19 and hospitalized between May 11 and November 28, 2020. We excluded pregnant patients and those who received sarilumab, baricitinib, or both drugs while hospitalized.

Procedure

The observation items in this study were information collected in the JA42434 study and obtained from case report forms in the COVIREGI-JP database [13]. Study data were collected and managed by the Joint Center for Researchers, Associates and Clinicians (JCRAC) data center of the National Center for Global Health and Medicine using REDCap [16, 17].

Endpoints, treatment strategies of interest, and follow-up

For the COVIREGI-JP cohorts, demographic information, epidemiological characteristics, comorbidities, signs, and symptoms were obtained during hospital admission (including conditions at admission). The outcome at discharge, as well as supportive care, history of drug administration, and complications during hospitalization were also collected. These data were collected on Days 1, 4, 8, 15, 22, and 29, starting from the date of admission [13].

In the JA42434 study, the following ordinal category scores were collected daily – 1: discharge from hospital or awaiting discharge; 2: oxygen administration unnecessary; 3: a cannula or mask, and reservoir, required for noninvasive oxygen administration; 4: bilevel positive airway pressure (BIPAP), continuous positive airway pressure (CPAP), or high flow required for noninvasive oxygen administration; 5: invasive mechanical ventilation required; 6: extracorporeal membrane oxygenation or invasive mechanical ventilation with additional organ support (renal replacement therapy, dialysis, cardiac stimulants, and/or vasoconstrictors) required; and 7: death [18]. For the COVIREGI-JP cohort tocilizumab and SOC sets, the clinical symptoms of each patient were classified into the above seven ordinal categories daily. This classification was conducted based on the data obtained during the study and the prespecified imputation algorithm (Supplementary Table 1; Additional file 1).

The primary endpoint was set as the time to discharge or the state of awaiting discharge (i.e., Category 1 on the seven-category ordinal scale). The secondary endpoints were as follows: (i) clinical state on the seven-category ordinal scale; (ii) time until improvement in clinical state on the seven-category ordinal scale; (iii) time until clinical failure (i.e., until the first occurrence of Category 5 or higher); (iv) time until recovery (i.e., until reaching Category 2); (v) mortality rate; (vi) proportion of patients using mechanical ventilators; and (vii) proportion of patients admitted to intensive-care units.

The follow-up time for each study was as follows: Each patient was observed until the earlier of the following: (i) Day 28 after the initial administration of the study drug (for the JP40959 study), hospitalization Day 29 from the date of admission (for the COVIREGI-JP cohort); (ii) death; and (iii) discharge or transfer from the hospital (only for the COVIREGI-JP cohort).

Statistical methods

The COVIREGI-JP cohort SOC set was matched in variable ratios (1:1 to 4:1) to the JA42434 study tocilizumab set for the primary study objective and to the COVIREGI-JP cohort tocilizumab set for the secondary objective. Matching was performed using exact matching, followed by propensity score matching. To considerably minimize immortal time bias [19], the origin time (study Day 1) and evaluation period for time-to-event endpoints were defined as follows: First, we defined hospitalization Day 1 as the hospital admission date and then defined study Day 1 for the tocilizumab sets (in both JA42434 and COVIREGI-JP) as the initial tocilizumab administration date. Study Day 1 for patients in the SOC groups was defined according to their matched participant from the tocilizumab sets. Specifically, we sequentially conducted matching based on the information obtained during hospitalization Day 1 through study Day 1 for each patient in the tocilizumab sets. If SOC participants were matched to a tocilizumab participant, the hospitalization day on which the matched tocilizumab participant received treatment was taken to be study Day 1 for the SOC participants. For example, if matching was performed with a patient in the tocilizumab set for which tocilizumab was initially administered on hospitalization Day 5, then hospitalization Day 5 was taken to be study Day 1 for the matched patient in the COVIREGI-JP cohort SOC set and used as the origin baseline timepoint. For all patients in the tocilizumab and SOC groups, the evaluation period was set from study Day 1 to hospitalization Day 29.

Participants with either decreased oxygen saturation (SpO2 ≤ 93%) or hypoxemia (PaO2/FiO2 < 300 mmHg) from hospitalization Day 1 to study Day 1 were selected for each matching, ensuring maximum consistency with JA42434. We then selected SOC participants via exact matching based on the following variables measured between hospitalization Day 1 and study Day 1: abnormal pulmonary signs shown by thoracic X-ray or CT scan, the seven-category ordinal scale, invasive mechanical ventilation, and therapeutic drugs and methods (oral or intravenous steroids, antiviral agents [remdesivir and favipiravir], and plasmapheresis or plasma exchange). Among the patients selected, at most four SOC participants with the closest propensity scores within the prespecified caliper (0.2 times the standard deviation of the logit-transformed value for the propensity score in the tocilizumab group) were matched to each tocilizumab participant. The propensity score for each subject used in matching was estimated using a logistic regression model. Possible study Day 1 for the SOC set varied across matching-candidate patients in the tocilizumab sets. Hence, we used the following risk factors assessed on hospitalization Day 1 to estimate the propensity score of each patient: age, sex, weight, the use of invasive mechanical ventilation, comorbidities that are risk factors for increased severity, comorbidities that are risk factors for death, and comorbidities that are risk factors for both increased severity and death.

The participants in SOC groups were weighted using the reciprocal numbers of SOC participants in their matched sets (i.e., weights ranged from 0.25 to 1). For analysis of the primary endpoint, hospitalization Day 28 was handled as the censoring time if mortality events occurred during the evaluation period. The groups were compared using Kaplan–Meier plots and the Cox regression model. Nominal P-values were calculated using the Wald method. In addition, subgroup analysis for the primary endpoint was pre-planned for the use or non-use of remdesivir (between hospitalization Day 1 and study Day 1) and steroids (between hospitalization Day 1 and study Day 1), and the score on the seven-category ordinal scale on study Day 1 (Categories 2 to 3 and 4 to 5). Among secondary endpoints, time until recovery or improvement in the clinical state indicated by the seven-category ordinal scale was handled similarly. We also conducted a sensitivity analysis to adjust for censoring in post-hoc analyses. Given the number of censored cases in each objective, different analyses were conducted for primary and secondary objectives. For the primary outcome, the time to discharge or the state of awaiting discharge, transfer from the hospital was handled as censoring for the COVIREGI-JP cohort as hospital transfers prevented follow-up. We estimated the bounds by imputing the “extreme” outcome times, i.e., imputing “event occurrence” as soon as censored or “event-free” until the end of the follow-up for each censored case. Two pattern imputations for the censoring time, the best and worst scenarios for the tocilizumab group, were conducted. For the secondary objective, an inverse probability of censoring weighted (IPCW) method was conducted by adjusting for the baseline and time-varying variables listed below.

All statistical analyses were performed using SAS version 9.4 (SAS Institute, Cary, NC, USA).

Results

Flow diagram

The flow diagram in Fig. 2 shows the procedure up to defining the analysis sets. At the data extraction time for the COVIREGI-JP cohort, 16,898 patients were included, including 372 in the COVIREGI-JP cohort tocilizumab set, these being patients administered tocilizumab while hospitalized, and 16,526 in the COVIREGI-JP cohort SOC set, these being patients not administered tocilizumab. In addition, there were 48 patients in the JA42434 study tocilizumab set, these being patients administered tocilizumab.

Fig. 2
figure 2

A flow diagram defining the analysis set for the study. ¶: Patients who meet the following JA42434 inclusion criteria: (i) evaluation performed between study Days –7 and 1; (ii) decreased oxygen saturation (SpO2 ≤ 93%) or hypoxemia (PaO2/FiO2 < 300 mmHg); and (iii) abnormal pulmonary signs shown by thoracic X-ray or CT scan. §: Same patient may be included in the SOC groups prepared for the primary and secondary objectives from the COVIREGI-JP cohort SOC set

Full size image

For the primary objective analysis set, matching was performed with patients from the JA42434 study tocilizumab set and the COVIREGI-JP cohort SOC set. The tocilizumab and SOC groups included 31 and 86 patients, respectively. The mean matching ratio for each subject in this analysis set was approximately 2.77. Table 1 shows the patient demographic characteristics of the analysis set for the primary objective.

Table 1 Comparison of patient demographic characteristics of the analysis set for the primary objective
Full size table

Weighted analysis was performed as appropriate to the matching ratio. For weighted analyses, numbers of participants are expressed to two decimal places.

The baseline characteristics for study Day 1 are provided in Table 2. In the analysis set for the primary objective, the proportion of men was approximately 86%, whereas the mean age was approximately 65 years. On study Day 1, approximately 70% of the participants had a score of 3 on the seven-category ordinal scale (i.e., a requirement other than for BIPAP/CPAP or high flow for noninvasive oxygen administration). In addition, the proportion of participants who used remdesivir was approximately 30%, and the median time after COVID-19 diagnosis was 4.0 days. It was confirmed that the groups had similar demographic characteristics, as well as previously administered and concomitant drugs.

Table 2 Comparison of patient baseline characteristics of analysis set for primary objective on study Day 1
Full size table

For the secondary objective analysis set, matching was performed using participant data from the COVIREGI-JP cohort tocilizumab and SOC sets. The tocilizumab and SOC groups included 116 and 411 subjects, respectively. The mean matching ratio in this analysis set was approximately 3.54. Patient demographic characteristics of the secondary objective analysis set are provided in Table 3.

Table 3 Comparison of patient demographic characteristics of the analysis set for secondary objective
Full size table

Weighted analysis was performed as appropriate to the matching ratio, and for weighted analyses, numbers of subjects are expressed to two decimal places.

Patient baseline characteristics of the analysis set for the secondary objective are provided in Table 4. The demographic characteristics were similar in the groups for the primary and secondary objectives.

Table 4 Comparison of patient baseline characteristics of analysis set for secondary objective at study Day 1
Full size table

Results for the primary endpoint

In the primary objective, the median time from study Day 1 to discharge or a state of awaiting discharge was 15 days in the JA42434 study tocilizumab set (95% confidence interval [CI]: 12–28 days) and 16 days in the SOC group (95% CI: 10 days to infinity). The hazard ratio (HR) for time until discharge from hospital or a state of awaiting discharge was 1.070 (95% CI: 0.565–2.028, P = 0.84). The Kaplan–Meier curves for time to discharge or a state of awaiting discharge are shown in Fig. 3A. The event occurrence rates by study Days 14 and 21, respectively, were 46.7% (95% CI: 30.9–65.7) and 65.5% (95% CI: 48.3–82.1) in the tocilizumab group and 43.1% (95% CI: 30.8–57.8) and 59.7% (95% CI: 46.5–73.3) in the SOC group (Table 5). The results of subgroup analysis were similar. The HRs obtained by sensitivity analysis for the censored cases were 1.315 (95% CI: 0.698–2.478, P = 0.40) and 0.845 (95% CI: 0.459–1.555, P = 0.59), respectively, for the best and worst scenarios for the tocilizumab group.

Fig. 3
figure 3

Kaplan–Meier curve for time to discharge or state of awaiting discharge. Panel A: Comparison between the tocilizumab and SOC groups for the primary objective. Panel B: Comparison between the tocilizumab and SOC groups for the secondary objective. Weighted analysis was performed as appropriate to the matching ratio, and the numbers of subjects are expressed in two decimal places for the weighted analyses

Full size image
Table 5 Comparison between tocilizumab and SOC groups for time to discharge or state of awaiting discharge
Full size table

For the secondary objective, the median time from study Day 1 until discharge from hospital or a state of awaiting discharge was 22 days in the tocilizumab group (95% CI: 15 days to infinity) and 18 days in the SOC group (95% CI: 16–22 days). The inter-group HR of the tocilizumab group to the SOC group for time until discharge from hospital or a state of awaiting discharge was 0.846 (95% CI: 0.582–1.230, P = 0.38).The Kaplan–Meier curves for time until discharge from hospital or a state of awaiting discharge are shown in Fig. 3B. The HR adjusting for censoring through the IPCW method was estimated as 0.842 (95% CI: 0.623–1.138, P = 0.26). In addition, the event occurrence rates by study Days 14 and 21 were 34.2% (95% CI: 25.4–44.9) and 48.5% (95% CI: 38.6–59.4) and 38.9% (95% CI: 33.8–44.5) and 55.5% (95% CI: 50.0–61.3) in the tocilizumab and SOC groups, respectively (Table 5). Similar results were obtained during subgroup analysis.

Results for secondary endpoints

The secondary endpoints were as follows: clinical symptoms according to the seven-category ordinal scale, time until improvement in clinical state, time until clinical failure, time until recovery, and mortality rate. No consistent trends between groups were found with these for either the primary or secondary objectives.

The distribution of clinical status using the seven-category ordinal scale for the primary objective is shown in Table 6. The median category scores for both the tocilizumab and SOC groups on study Days 14, 21, and 28 were 2.0, 1.0, and 1.0, respectively. The median time until improvement in clinical state was 13 days and 16 days for the tocilizumab and SOC groups, respectively (HR 1.112, 95% CI: 0.596–2.075, P = 0.74). The HRs for time to clinical failure or time to recovery were 0.628 (95% CI: 0.202–1.953, P = 0.42) and 1.019 (95% CI: 0.555–1.871, P = 0.95), respectively. Furthermore, the mortality rate in the tocilizumab and SOC groups was 9.7% and 19.6%, respectively. The estimated risk difference was –9.9% (95% CI: –28.0–9.4).

Table 6 Distribution of the clinical status for the primary objective on study Days 14, 21, 28
Full size table

The distribution of clinical status for the secondary objective is also shown in Table 6. The HRs for time until improvement in clinical state, time until clinical failure, and time until recovery were estimated as 0.844 (95% CI: 0.598–1.190, P = 0.33), 1.670 (95% CI: 0.981–2.844, P = 0.06), and 0.802 (95% CI: 0.567–1.136, P = 0.21), respectively. The risk difference for the mortality rate was 4.3% (95% CI: –5.2–13.8). Moreover, there were no differences between the groups.

Discussion

This retrospective cohort study was designed to evaluate the efficacy of tocilizumab over the SOC for patients with severe COVID-19-related pneumonia in two different databases: the JA42434 single-arm study and the COVIREGI-JP data. A synthetic control group from the COVIREGI-JP cohort was utilized as a benchmark for the tocilizumab group. In this study, tocilizumab did not improve the time to discharge or the state of awaiting discharge. With respect to clinical symptoms based on the seven-category ordinal scale, time to improvement in clinical state, time to clinical failure, time to recovery, and mortality rate, no consistent improvement trend of tocilizumab was found for either the primary or secondary objectives.

Our study findings are inconsistent with those of previous trials that showed tocilizumab to be an effective treatment over the SOC for hospitalized patients with COVID-19 characterized by hypoxia [8, 20,21,22]. Results from the RECOVERY trial demonstrated that patients allocated to the tocilizumab group had a lower 28-day mortality rate than those who received standard medical care alone (621 [31%] of 2022 patients in the tocilizumab group vs 729 (35%) of 2094 patients in the SOC group; rate ratio 0.85; 95% CI, 0.76–0.94; P = 0.0028). Moreover, tocilizumab was associated with a greater probability of discharge from hospital within 28 days (57% vs 50%) [8]. The REMAP-CAP trial recruited critically ill patients within 24 h of organ support initiation and showed tocilizumab to be more effective than standard care in reducing the risk for continued organ support. This treatment protocol prolonged survival in critically ill patients within 24 h of starting organ support [22]. Furthermore, meta-analyses have demonstrated that tocilizumab reduced all-cause mortality by Day 28 compared with standard care alone [23, 24].

Our findings are consistent with those of another randomized clinical trial involving hospitalized adult patients with COVID-19 pneumonia and a Pao2/Fio2 ratio of 200–300 mmHg. Patients who received tocilizumab in the COVINTOC trial showed no benefit in terms of disease progression compared to those on standard care [25]. The COVINTOC trial reported that the primary endpoint, namely, the proportion of patients with progression of COVID-19 from moderate to severe or from severe to death up to Day 14, was not significantly different between the tocilizumab plus standard care and standard care alone groups. Similarly, the secondary endpoints that included time to clinical improvement according to COVID-19 grade and time to hospital discharge did not show any significant difference between the two groups [26]. Additionally, although a few retrospective studies in Japan have evaluated the efficacy of tocilizumab using real-world data, none of them demonstrated a significant efficacy of tocilizumab over the SOC [27, 28].

Here, the efficacy of tocilizumab compared to the SOC could not be demonstrated in terms of primary and secondary endpoints. Attempts to use real-world data as an external synthetic control for drug evaluation in single-arm clinical trials have received considerable interest in recent years [29,30,31]. We designed and analyzed the study based on the points discussed therein. However, there are several limitations with respect to the study results, and care is needed with result interpretation.

  1. 1)

    Limitations in the comparability of the two groups

The patients enrolled in the JA42434 study showed signs of pneumonia from study Days –2 to 1, with decreased oxygen saturation (SpO2 ≤ 93%) or hypoxemia (PaO2/FiO2 < 300 mmHg). Owing to difficulties with the stipulated evaluation time for the COVIREGI-JP cohort, patients who showed signs of pneumonia on study Days –7 to 1 were selected. Therefore, patients in the COVIREGI-JP cohort could have been selected without showing signs of pneumonia on study Days –2 to 1, which likely impacted the HR for the time to discharge or a state of awaiting discharge up to Days 7. For the tocilizumab group, study Day 1 was defined as the date of initial tocilizumab administration. In contrast, study Day 1 in the SOC group was defined using the hospitalization date of the matched tocilizumab participant. The hospitalization day of the matched tocilizumab patient was considered study Day 1 for the SOC participant. Although it would have been ideal to have included laboratory data on study Day 1 to match background factors, as the COVIREGI-JP cohort did not have sufficient laboratory data for all patients and the laboratory data collected after admission were limited, the study was not designed to include laboratory data for matching. Therefore, it is possible that confounding bias between the two groups of this study was not completely eliminated. To achieve more similar distributions of adjusted risk factors between the two groups, we narrowed the matching-caliper coefficient to 0.1. However, similar results were obtained for both the primary and secondary endpoints. The results of this sensitivity analysis were natural in the case where there were insufficient data to adjust risk factors.

  1. 2)

    Censored data, including subjects for whom follow-up was impossible

In the JA42434 study, progression could be followed up to Day 28, even after hospital transfers, except in patients who withdrew from the study. In the COVIREGI-JP cohort, hospital transfers could prevent follow-up of progression, which would affect evaluation. Thus, subjects for whom “the patient’s condition deteriorated and he/she was transferred to a different hospital” or “the patient’s condition improved and he/she was transferred to a different hospital” was noted may have led to selection bias. This may have led to the discrepancy in time to failure results between the primary and secondary objectives. Although we conducted sensitivity analyses using the bound approach for the primary analysis and the IPCW method for the secondary analysis, the interpretation of the results may be limited. As data on the conditions of patients immediately before censoring were not available, the selection bias may not have been adequately adjusted.

  1. 3)

    Matching

Although the target numbers of participants for the primary and secondary objectives in the SOC group were four times the respective numbers in the tocilizumab group, the number of subjects meeting the conditions for matching did not reach the expected number. Therefore, the number of participants included in the analysis was lower than the target number. Caliper matching using the propensity score and exact matching method was performed to increase the comparability between the two groups despite the smaller number of study participants included in the analysis.

Conclusions

The effect of tocilizumab against COVID-19 pneumonia was evaluated in this retrospective cohort study. Our results indicated that tocilizumab treatment had no effect on outcomes, including on time to discharge. This study had several limitations, mostly due to the use of a synthetic control from the registry data.

Availability of data and materials

The datasets generated and/or analyzed during the current study are not publicly available because of a contract with Chugai Pharmaceutical Co., but are available from the corresponding author upon reasonable request and with permission of the data owner.

Abbreviations

CI:

Confidence interval

COVIREGI-JP:

COVID-19 Registry Japan

CT:

Computed tomography

HR:

Hazard ratio

SOC:

 Standard of care

X-ray:

Chest X-radiography

References

  1. Aziz M, Fatima R, Assaly R. Elevated interleukin-6 and severe COVID-19: a meta-analysis. J Med Virol. 2020;92:2283–5. https://doi.org/10.1002/jmv.25948.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Giamarellos-Bourboulis EJ, Netea MG, Rovina N, Akinosoglou K, Antoniadou A, Antonakos N, et al. Complex immune dysregulation in COVID-19 patients with severe respiratory failure. Cell Host Microbe. 2020;27:992–1000.e3. https://doi.org/10.1016/j.chom.2020.04.009.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Hashizume M. Outlook of IL-6 signaling blockade for COVID-19 pneumonia. Inflam Regen. 2020;40:24. https://doi.org/10.1186/s41232-020-00134-7.

    Article  CAS  Google Scholar 

  4. Ruan Q, Yang K, Wang W, Jiang L, Song J. Clinical predictors of mortality due to COVID-19 based on an analysis of data of 150 patients from Wuhan China. Intensive Care Med. 2020;46:846–8. https://doi.org/10.1007/s00134-020-05991-x.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Zhu J, Pang J, Ji P, Zhong Z, Li H, Li B, et al. Elevated interleukin-6 is associated with severity of COVID-19: a meta-analysis. J Med Virol. 2021;93:35–7.

    Article  CAS  PubMed  Google Scholar 

  6. Holt GE, Batra M, Murthi M, Kambali S, Santos K, Bastidas MVP, et al. Lack of tocilizumab effect on mortality in COVID19 patients. Sci Rep. 2020;10:17100. https://doi.org/10.1038/s41598-020-74328-x.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Capra R, De Rossi N, Mattioli F, Romanelli G, Scarpazza C, Sormani MP, et al. Impact of low dose tocilizumab on mortality rate in patients with COVID-19 related pneumonia. Eur J Intern Med. 2020;76:31–5. https://doi.org/10.1016/j.ejim.2020.05.009.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Abani O, Abbas A, Abbas F, Abbas M, Abbasi S, Abbass H, et al. Tocilizumab in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial. Lancet. 2021;397:1637–45. https://doi.org/10.1016/S0140-6736(21)00676-0.

    Article  Google Scholar 

  9. Xu X, Han M, Li T, Sun W, Wang D, Fu B, et al. Effective treatment of severe COVID-19 patients with tocilizumab. Proc Natl Acad Sci U S A. 2020;117:10970–5. https://doi.org/10.1073/pnas.2005615117.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Toniati P, Piva S, Cattalini M, Garrafa E, Regola F, Castelli F, et al. Tocilizumab for the treatment of severe COVID-19 pneumonia with hyperinflammatory syndrome and acute respiratory failure: a single center study of 100 patients in Brescia Italy. Autoimmun Rev. 2020;19: 102568. https://doi.org/10.1016/j.autrev.2020.102568.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Sciascia S, Aprà F, Baffa A, Baldovino S, Boaro D, Boero R, et al. Pilot prospective open, single-arm multicentre study on off-label use of tocilizumab in patients with severe COVID-19. Clin Exp Rheumatol. 2020;38:529–32.

    PubMed  Google Scholar 

  12. Rajkumar RP. Cross-national variations in COVID-19 mortality: the role of diet, obesity and depression. Diseases. 2021;9:36. https://doi.org/10.3390/diseases9020036.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Matsunaga N, Hayakawa K, Terada M, Ohtsu H, Asai Y, Tsuzuki S, et al. Clinical epidemiology of hospitalized patients with coronavirus disease 2019 (COVID-19) in Japan: Report of the COVID-19 registry Japan. Clin Infect Dis. 2021;73:e3677–89. https://doi.org/10.1093/cid/ciaa1470.

    Article  CAS  PubMed  Google Scholar 

  14. Kamegai K, Hayakawa K. Towards the light at the end of the tunnel: Changes in clinical settings and political measures regarding COVID-19 from 2021, and future perspectives in Japan. Glob Health Med. 2022;4:327–31. https://doi.org/10.35772/ghm.2022.01071.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Thorlund K, Dron L, Park JJH, Mills EJ. Synthetic and external controls in clinical trials - A primer for researchers. Clin Epidemiol. 2020;8(12):457–67. https://doi.org/10.2147/CLEP.S242097.

    Article  Google Scholar 

  16. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research Electronic Data Capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377–81. https://doi.org/10.1016/j.jbi.2008.08.010.

    Article  PubMed  Google Scholar 

  17. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O’Neal L, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform. 2019;95:103208. https://doi.org/10.1016/j.jbi.2019.103208.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Rosas IO, Bräu N, Waters M, Go RC, Hunter BD, Bhagani S, et al. Tocilizumab in hospitalized patients with severe Covid-19 pneumonia. N Engl J Med. 2021;384:1503–16. https://doi.org/10.1056/NEJMoa2028700.

    Article  CAS  PubMed  Google Scholar 

  19. Suissa S, Dell’Aniello S. Time-related biases in pharmacoepidemiology. Pharmacoepidemiol Drug Saf. 2020;29:1101–10. https://doi.org/10.1002/pds.5083.

    Article  PubMed  Google Scholar 

  20. Rosas IO, Bräu N, Waters M, Go RC, Malhotra A, Hunter BD, et al. Tocilizumab in patients hospitalised with COVID-19 pneumonia: Efficacy, safety, viral clearance, and antibody response from a randomised controlled trial (COVACTA). EClinicalmedicine. 2022;47:101409. https://doi.org/10.1016/j.eclinm.2022.101409.

    Article  PubMed  PubMed Central  Google Scholar 

  21. WHO Rapid Evidence Appraisal for COVID-19 Therapies (REACT) Working Group, Sterne JAC, Murthy S, Diaz JV, Slutsky AS, Villar J, et al. Association Between Administration of Systemic Corticosteroids and Mortality Among Critically Ill Patients With COVID-19: A Meta-analysis. JAMA. 2020;324:1330–41. https://doi.org/10.1001/jama.2020.17023.

    Article  CAS  Google Scholar 

  22. REMAP-CAP Investigators, Gordon AC, Mouncey PR, Al-Beidh F, Rowan KM, Nichol AD, et al. Interleukin-6 receptor antagonists in critically ill patients with Covid-19. N Engl J Med. 2021;384:1491–502. https://doi.org/10.1056/NEJMoa2100433.

    Article  Google Scholar 

  23. Godolphin PJ, Fisher DJ, Berry LR, Derde LPG, Diaz JV, Gordon AC, et al. Association between tocilizumab, sarilumab and all-cause mortality at 28 days in hospitalised patients with COVID-19: A network meta-analysis. PLOS One. 2022;17:e0270668. https://doi.org/10.1371/journal.pone.0270668.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Boregowda U, Perisetti A, Nanjappa A, Gajendran M, Kutti Sridharan G, Goyal H. Addition of tocilizumab to the standard of care reduces mortality in severe COVID-19: a systematic review and meta-analysis. Front Med (Lausanne). 2020;7:586221. https://doi.org/10.3389/fmed.2020.586221.

    Article  PubMed  Google Scholar 

  25. Salvarani C, Dolci G, Massari M, Merlo DF, Cavuto S, Savoldi L, et al. Effect of tocilizumab vs standard care on clinical worsening in patients hospitalized with COVID-19 pneumonia: a randomized clinical trial. JAMA Intern Med. 2021;181:24–31. https://doi.org/10.1001/jamainternmed.2020.6615.

    Article  CAS  PubMed  Google Scholar 

  26. Soin AS, Kumar K, Choudhary NS, Sharma P, Mehta Y, Kataria S, et al. Tocilizumab plus standard care versus standard care in patients in India with moderate to severe COVID-19-associated cytokine release syndrome (COVINTOC): an open-label, multicentre, randomised, controlled, phase 3 trial. Lancet Respir Med. 2021;9:511–21. https://doi.org/10.1016/S2213-2600(21)00081-3.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Kawamata T, Tanino Y, Nikaido T, Minemura H, Sato Y, Togawa R, et al. Clinical effect of early administration of tocilizumab following the initiation of corticosteroid therapy for patients with COVID-19. J Infect Chemother. 2022;28:1639–44.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Mitsushima S, Horiguchi H, Taniguchi K. Effectiveness of drugs for COVID-19 inpatients in Japanese medical claim data as average treatment effects with inverse probability weighted regression adjustment: Retrospective observational study. [Preprint]

  29. United States Food & Drug Administration, Framework for FDA’s Real-World Evidence Program. https://www.fda.gov/media/120060/download. Accessed Feb 2023.

  30. Duke Margolis Center for Health Policy. Understanding the need for non-interventional studies using secondary data to generate real-world evidence for regulatory decision making, and demonstrating their credibility. 2019. https://healthpolicy.duke.edu/sites/default/files/2020-08/Non-InterventionalStudyCredibility.pdf . Accessed Feb 2023.

  31. Chen J, Ho M, Lee K, Song Y, Fang Y, Goldstein BA, et al. The current landscape in biostatistics of real-world data and evidence: clinical study design and analysis. Stat Biopharm Res. 2023;15:29–42. https://doi.org/10.1080/19466315.2021.1883474.

    Article  Google Scholar 

Download references

Acknowledgements

Editorial support, in the form of medical writing, assembling tables and creating high-resolution images based on the authors’ detailed directions, collating author comments, copyediting, fact checking, and referencing, was provided by Editage, Cactus Communications, and funded by Chugai Pharmaceutical Co.

Funding

This study was conducted as collaborative research funded by Chugai Pharmaceutical Co. and was supported by the Health and Labor Sciences Research Grant “Research on Emerging and Re-emerging Infectious Diseases and Immunization” Program (19HA1003). A fiduciary agreement will be concluded upon receipt of the funding. The term of the contract is from the approval of the Ethical Review Committee to June 30, 2023.

Author information

Authors and Affiliations

  1. Biostatistics Section, Department of Data Science, Center for Clinical Sciences, National Center for Global Health and Medicine, 1-21-1 Toyama, Shinjyuku-ku, Tokyo, 162-8655, Japan

    Yukari Uemura & Yousuke Shimizu

  2. Biometrics Department, Clinical Development Division, Chugai Pharmaceutical CO., LTD, Tokyo, Japan

    Ryoto Ozaki

  3. Department of Information and Computer Technology, Faculty of Engineering, Tokyo University of Science, Tokyo, Japan

    Tomohiro Shinozaki

  4. Clinical Pharmacology and Regulatory Sciences, Juntendo University Graduate School of Medicine, Tokyo, Japan

    Hiroshi Ohtsu

  5. Center for Clinical Sciences, National Center for Global Health and Medicine, Tokyo, Japan

    Kazuo Izumi

  6. Disease Control and Prevention Center, National Center for Global Health and Medicine, Tokyo, Japan

    Sho Saito, Nobuaki Matsunaga & Norio Ohmagari

Authors
  1. Yukari Uemura
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ryoto Ozaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tomohiro Shinozaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hiroshi Ohtsu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yousuke Shimizu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kazuo Izumi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sho Saito
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Nobuaki Matsunaga
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Norio Ohmagari
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

YU – wrote the main manuscript, planned the statistical methods, and analyzed and interpreted the results; TS – planned the statistical methods and analyzed and interpreted the results; RO – planned the study and the statistical methods and analyzed and interpreted the results; HO – planned the statistical methods; YS – interpreted the results; KI – planned the study; SS – planned the study and interpreted the results; NM – planned the study and interpreted the results; NO – planned the study and interpreted the results. All authors reviewed the manuscript.

Corresponding author

Correspondence to Yukari Uemura.

Ethics declarations

Ethics approval and consent to participate

This research was conducted in accordance with ethical principles based on the Declaration of Helsinki and in compliance with the ethical guidelines for medical research involving human subjects. This study was approved by the National Center for Global Health and Medicine Ethics Review Committee (NCGM-G-004119-00) and registered in April 2020 in the UMIN Clinical Trials Registry (UMIN-CTR; ID: UMIN000043622). Informed consent from each patient was obtained in the form of opt-out. Information regarding opting out of our study is available on the registry website. The participants of the source study, JA42434 study, provided informed consent.

Consent for publication

Not applicable

Competing interests

YU received a consultation fee from Chugai Pharmaceutical Ltd, RO is an employee of Chugai Pharmaceutical Ltd. Rest all other author's have no competing interest

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Supplementary Table 1.

The pre-specified imputation algorithm for the seven-category ordinal scale.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Uemura, Y., Ozaki, R., Shinozaki, T. et al. Comparative effectiveness of tocilizumab vs standard care in patients with severe COVID-19-related pneumonia: a retrospective cohort study utilizing registry data as a synthetic control. BMC Infect Dis 23, 849 (2023). https://doi.org/10.1186/s12879-023-08840-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12879-023-08840-6

Keywords

BMC Infectious Diseases

ISSN: 1471-2334

Contact us