Skip to main content

Issues with reporting and interpretation of Khan et al. 2021

A Research article to this article was published on 27 September 2021

Abstract

A recent publication in BMC Infectious Diseases concerning the use of convalescent plasma for the treatment of COVID-19 had a number of major issues. This correspondence details specific instances of unclear reporting as well as major omissions when discussing the context of the trial. These render the study's findings and conclusions misleading.

Peer Review reports

Background

The published study from Khan and colleagues[1] on the treatment of COVID-19 patients in Pakistan with convalescent plasma (CP) contains a number of serious reporting issues. Unclear reporting and outcome definitions, failure to adequately discuss trial limitations, and the omission of large volumes of relevant literature from the discussion section render the study’s findings and conclusions misleading.

Main text

The authors provide two trial registrations for this clinical trial: NCT04352751 on ClinicalTrials.gov; and IRCT20200414047072N1 on the Iranian Registry of Clinical Trials (IRCT). Trial registrations should be prospective to ensure no post hoc deviations can occur. The authors state that the first plasma infusions were given on 4 April 2020, however both trial registrations were submitted after the trial started: to ClinicalTrials.gov on 16 April 2020 and to IRCT on 28 April 2020. While BMC editorial policy does allow for the publication of retrospectively registered studies, it must be declared. The authors note the registration date was 28 April 2020 but do not make clear that this was retrospective [2].

Comparing the target sample size with the number of trial participants included raises additional concerns. The published report includes 50 COVID-19 patients, however the IRCT entry claims that this trial aimed for a target sample size of 357 and the ClinicalTrials.gov entry notes an estimated enrollment of 2000. The authors make no mention of the current recruitment status of the trial, whether further enrollment is planned, nor whether the trial was terminated early. Both registries offer conflicting information on the trial’s recruitment status: ClinicalTrials.gov says the trials is “Recruiting” but the IRCT states “Recruitment complete”. Neither registry entry has been updated since 2020. With no additional context or explanation offered, readers are left to wonder why the sample size is dramatically lower than planned, and if this compromises the interpretation of the results.

The outcomes reported in this study are inadequately defined, and different from those pre-specified on both ClinicalTrials.gov and IRCT. The published report states: "The primary and secondary endpoint of this study were, (a) improvement in clinical signs and symptoms and change in category of disease severity, (b) ability of CP to halt disease progression leading to invasive ventilation, respectively”, with no further details given in the report on the categories of disease severity nor the time-points at which outcomes were assessed. On ClinicalTrials.gov, the authors pre-specify one adequately defined primary outcome (“Change in COVID-19 severity status”) but do not pre-specify the secondary outcome that was reported. The IRCT contains five pre-specified primary outcomes: of these, four are inadequately pre-specified, with only vague detail given on the method of measurement for each outcome. While some of these overlap with the components of the primary outcome on ClinicalTrials.gov, it is not stated whether these pre-specified outcomes on the IRCT relate to that primary outcome, and the secondary outcome in the published article remains unspecified in the IRCT entry. The outcome reporting in this study therefore falls well short of the Consolidated Standards of Reporting Trials (CONSORT) reporting standards, which are endorsed by BMC Infectious Diseases, that state trial reports should include "Completely defined pre-specified primary and secondary outcome measures, including how and when they were assessed" and that authors should report “Any changes to trial outcomes after the trial commenced, with reasons.” [3, 4]

Lastly, the interpretation of this study is notably incomplete. The authors concede that the lack of a control group is a limitation in their study noting that “no inference on efficacy of CP can be drawn”, however they then conclude that plasma transfusion “seemed to be the safest modality of managing COVID-19 patients and the hall mark[sic] of this therapy is timely infusion and selection of category of recipients.” They cite only a single positive non-randomised study as context for their findings [5]. BMC Infectious Diseases accepted this study on 20 July 2021. The authors failed to consider either the Cochrane review (last updated May 2021) or a major review published in the Journal of the American Medical Association in February 2021, both of which examined randomized evidence and concluded, with confidence, that convalescent plasma “does not reduce mortality and has little to no impact on measures of clinical improvement” [6, 7]. In fact, it appears none of the 13 studies included in the Cochrane review, which included a total of 48,509 trial participants, are cited anywhere, including the landmark results from the RECOVERY trial [8]. All of these would have been available to the authors, editors and reviewers prior to acceptance. Readers are left with the impression that plasma is an effective therapy when substantial evidence to the contrary was left uncited [9].

Conclusion

Overall, the reporting and interpretation of this study are extreme examples of deviations from best practice. If trial reports do not disclose and explain deviations from registrations, they can present readers with a misleading picture of the true results of the trial. Where trial outcomes are vaguely defined, the research community is unable to critically appraise or reproduce findings. Finally, the omission of large volumes of supporting evidence prevents readers from understanding the context of the reported trial, and allows for erroneous and misleading conclusions about a treatment’s efficacy and safety.

Availability of data and materials

Not applicable. This comment involved no original data or materials.

Abbreviations

CP:

Convalescent plasma

IRCT:

Iranian Registry of Clinical Trials

CONSORT:

Consolidated Standards of Reporting Trials

References

  1. Khan TNS, Mukry SN, Masood S, Meraj L, Devrajani BR, Akram J, et al. Usefulness of convalescent plasma transfusion for the treatment of severely ill COVID-19 patients in Pakistan. BMC Infect Dis. 2021;21:1014.

    CAS  Article  Google Scholar 

  2. Editorial policies. BMC. https://www.biomedcentral.com/getpublished/editorial-policies. Accessed 3 Feb 2022.

  3. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340: c869.

    Article  Google Scholar 

  4. CONSORT—Endorsers. CONSORT. http://www.consort-statement.org/about-consort/endorsers. Accessed 21 Aug 2020.

  5. Liu STH, Lin H-M, Baine I, Wajnberg A, Gumprecht JP, Rahman F, et al. Convalescent plasma treatment of severe COVID-19: a propensity score-matched control study. Nat Med. 2020;26:1708–13.

    CAS  Article  Google Scholar 

  6. Piechotta V, Iannizzi C, Chai KL, Valk SJ, Kimber C, Dorando E, et al. Convalescent plasma or hyperimmune immunoglobulin for people with COVID-19: a living systematic review. Cochrane Database Syst Rev. 2021. https://doi.org/10.1002/14651858.CD013600.pub4.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Janiaud P, Axfors C, Schmitt AM, Gloy V, Ebrahimi F, Hepprich M, et al. Association of convalescent plasma treatment with clinical outcomes in patients with COVID-19: a systematic review and meta-analysis. JAMA. 2021;325:1185–95.

    CAS  Article  Google Scholar 

  8. Abani O, Abbas A, Abbas F, Abbas M, Abbasi S, Abbass H, et al. Convalescent plasma in patients admitted to hospital with COVID-19 (RECOVERY): a randomised controlled, open-label, platform trial. Lancet. 2021;397:2049–59.

    Article  Google Scholar 

  9. Gøtzsche PC. Citation bias: questionable research practice or scientific misconduct? J R Soc Med. 2022;115:31–5.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

There was no specific funding for this work.

Author information

Authors and Affiliations

Authors

Contributions

NJD conceptualized the letter and wrote the first draft. Both authors (NJD, HD) contributed equally to the analysis presented and substantially contributed to the final revisions. Both authors approve the final version of the manuscript and agree to be personally accountable for their contributions. Both authors read and approved the final manuscript.

Corresponding author

Correspondence to Nicholas J. DeVito.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests regarding this Correspondence.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

DeVito, N.J., Drysdale, H. Issues with reporting and interpretation of Khan et al. 2021. BMC Infect Dis 22, 567 (2022). https://doi.org/10.1186/s12879-022-07551-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12879-022-07551-8

Keywords

  • Trials Transparency
  • Trial Reporting
  • Trial Registration
  • Convalescent Plasma
  • Covid-19