Skip to main content

Clinical application of cell-free next-generation sequencing for infectious diseases at a tertiary children’s hospital

Abstract

Background

Children affected by infectious diseases may not always have a detectable infectious etiology. Diagnostic uncertainty can lead to prolonged hospitalizations, inappropriately broad or extended courses of antibiotics, invasive diagnostic procedures, and difficulty predicting the clinical course and outcome. Cell-free plasma next-generation sequencing (cfNGS) can identify viral, bacterial, and fungal infections by detecting pathogen DNA in peripheral blood. This testing modality offers the ability to test for many organisms at once in a shotgun metagenomic approach with a rapid turnaround time. We sought to compare the results of cfNGS to conventional diagnostic test results and describe the impact of cfNGS on clinical care in a diverse pediatric population at a large academic children’s hospital.

Methods

We performed a retrospective chart review of hospitalized subjects at a tertiary pediatric hospital to determine the diagnostic yield of cfNGS and its impact on clinical care.

Results

We describe the clinical application of results from 142 cfNGS tests in the management of 110 subjects over an 8-month study period. In comparison to conventional testing as a reference standard, cfNGS was found to have a positive percent agreement of 89.6% and negative percent agreement of 52.3%. Furthermore, 32.4% of cfNGS results were directly applied to make a clinical change in management.

Conclusions

We demonstrate the clinically utility of cfNGS in the management of acutely ill children. Future studies, both retrospective and prospective, are needed to clarify the optimal indications for testing.

Peer Review reports

Introduction

Children affected by infectious diseases may not always have a detectable infectious etiology. Diagnostic uncertainty can lead to prolonged hospitalizations, inappropriately broad or extended courses of antibiotics, invasive diagnostic procedures, and difficulty predicting the clinical course and outcome.

Cell-free plasma next-generation sequencing (cfNGS) can identify viral, bacterial, and fungal infections by detecting pathogen DNA in peripheral blood. This testing modality offers the ability to test for over 1000 organisms at once in a metagenomic approach with a rapid turnaround time of approximately 48 h [1,2,3]. This technique has successfully identified pathogens in cases with antibiotic pre-exposure and in deep-seated infections that traditionally require invasive sampling of infected tissue [2, 4].

The clinical use of cfNGS has been evaluated in adults with sepsis, oncology patients, and healthy children with community acquired pneumonia [1, 4, 5]. There is a growing body of evidence reporting the clinical application of cfNGS. While some studies demonstrate promising diagnostic applications when compared to conventional diagnostic testing, recent studies have found limited clinical impact when test results are applied directly to patient care [6,7,8,9].

In this study, we sought to compare the results of cfNGS to conventional diagnostic test results and describe the impact of cfNGS on clinical care in a diverse pediatric population at a tertiary, academic children’s hospital.

Methods

Subject population

This is a single-center, retrospective review of subjects admitted at a tertiary children’s hospital between June 1, 2017 and January 22, 2018 for whom cfNGS was included as part of their diagnostic workup. The Institutional Review Board (IRB) of Rady Children’s Hospital San Diego (RCHSD) and the Human Research Protections Program at the University of California San Diego approved this study. All methods were performed in accordance with the relevant guidelines and regulations. A waiver of informed consent was granted by the IRB of Rady Children’s Hospital San Diego and the Human Research Protections Program at the University of California San Diego for this retrospective study.

Cell free next generation sequencing

Peripheral blood samples were collected in a BD Vacutainer Plasma Preparation and sent to the Karius CLIA laboratory (Redwood City, CA) for next-generation sequencing. Sample analysis techniques have been previously described [1]. All subjects from whom cfNGS was obtained had pediatric infectious disease consultation, which guided the decision to obtain cfNGS. Our institution does not have standardized criteria for ordering cfNGS. The decision to obtain cfNGS testing is at the discretion of the consulting infectious disease attending.

Chart review process

Charts were reviewed to obtain objective subject information. Study data were collected and managed using the Research Electronic Data Capture (REDCap) electronic data capture tools hosted at Rady Children’s Hospital, San Diego, CA. REDCap is a secure, web-based application designed to support data capture for research studies.

Comparing conventional diagnostic test results

The results of cfNGS were compared to all standard-of-care diagnostic tests obtained within 3 days before or after cfNGS testing in order to better ensure that the comparison of cfNGS to conventional testing reflected a similar clinical picture. All non-cfNGS diagnostic tests sent during this period were considered “conventional diagnostic testing,” and included any diagnostic testing available to our institution including cultures from blood, respiratory aspirate, body fluid, and tissue samples in addition to targeted, pathogen-specific real-time PCR studies (from blood, cerebral spinal fluid, or respiratory swabs).

An organism identified by either conventional diagnostic testing or cfNGS was determined to be “pathologically significant” if the clinical team indicated that the result was plausible, if knowledge of the positive test was employed to make a clinical change in patient management, or retrospectively by an adjudication review of an infectious disease physician independent of the treating physicians. For organisms identified by cfNGS, each organism was separately adjudicated as a true positive (TP), false positive (FP), true negative (TN), or false negative (FN). The cfNGS test was than adjudicated as a single TP, FP, TN, or FN result, reflecting an aggregate assessment of the overall test result in the clinical context of that subject. cfNGS as a whole was adjudicated as TP if at least one organism identified by cfNGS was concordant with conventional testing or if at least one organism was deemed sufficient to the clinical scenario for that subject. Conventional testing was adjudicated as an aggregate of all conventional tests sent per subject. A reference standard was then generated from the adjudication of the conventional testing as pathogenic or not by an independent review panel. This application of a reference standard has been previously described [1, 6, 9]. We used this composite reference standard to estimate positive percent agreement (PPA) and negative percent agreement (NPA).

Clinical application of cfNGS results

Clinical changes to patient management made in response to cfNGS results were assessed by manual chart review. Additionally, a retrospective survey was completed for each cfNGS test by the infectious disease physician who ordered the test to assess the clinical impact. If this physician was not available, then one of five infectious disease faculty at RCHSD completed the survey. The objective chart review and subjective surveys were compared and evaluated by a third infectious disease physician to ensure concordance.

Results

Subjects

A total of 110 subjects over 111 individual hospitalizations met inclusion criteria. A total of 142 discrete cfNGS tests were obtained (18 subjects had more than one cfNGS tests sent during their hospitalization).

The most common admission diagnoses fell under infectious, respiratory, or hematological categories. Examples within these categories include sepsis, pneumonia, and febrile neutropenia. There is overlap between diagnostic categories of admission diagnosis. The most common co-morbidities at time of admission include immunocompromised status, pre-existing oncologic diagnoses, and the presence of a pre-existing central line. Over half (53.2%) of the subjects were admitted to an intensive care setting (neonatal, pediatric, or cardiac intensive care units) during their hospitalization (Table 1).

Table 1 Demographics

Conventional diagnostic test results

Conventional diagnostic tests obtained during the study period for this cohort of subjects included 224 blood cultures, 142 respiratory pathogen PCR tests, 133 body fluid culture (includes urine and pleural fluid), 97 respiratory cultures from endotracheal tubes or tracheostomies and 402 other tests (Supplemental Table 1). Only 17.1% of the 998 conventional diagnostic tests that were sent returned a positive result, which includes contaminants and false positives.

cfNGS results

The most common indication for cfNGS testing was clinical symptoms of infection followed by focal imaging findings (Table 2). The median day on which cfNGS was obtained was hospitalization day 6 (IQR 3–6). The median time to cfNGS result availability was 72 h (IQR 48–104). There were 105 tests that identified at least one infectious organism, 59 of which identified more than one organism.

Table 2 cfNGS Test Profile

When comparing cfNGS results to conventional testing, we found that of the 105 positive cfNGS tests, 27 (25.7%) identified the same organism as conventional tests. There were 92 tests that identified one or more additional organisms not previously identified by conventional diagnostic testing. This includes the 51 cfNGS tests that identified an organism while all conventional testing was negative (Supplemental Table 2).

Twenty-two cfNGS tests were negative and agreed with negative conventional testing. There were four negative cfNGS tests with no conventional testing available for comparison. There were 11 cfNGS tests that were negative when conventional testing identified an infectious source.

Interpretation and clinical application

Of the positive cfNGS tests, 69 (65.7%) were considered clinically relevant and were considered true positives. We calculated PPA and NPA data of cfNGS by comparison against the composite reference derived from adjudication of conventional testing as previously described [1, 6]. Conventional diagnostic testing had a PPA of 42% whereas cfNGS had a PPA of 89.6%. The NPA of conventional diagnostic testing was higher than that of cfNGS at 83.9 and 52.3% respectively (Supplemental Table 3).

Chart review determined that 32.4% of cfNGS results were directly applied to make a clinical change in management. This number includes the clinical application of eight tests with negative results. The most common clinical changes were changes to antimicrobial medication regimen: narrowed broad spectrum coverage in 46.7% and shortened duration of treatment in 28.9%. One subject underwent pneumonectomy because of Rhizopus oryzae identified on cfNGS (later confirmed by culture of resected lung tissue). The ordering physician reported that the results of the cfNGS was helpful in guiding patient care for 52.1% of the tests sent, which was higher than the percentage change of management (Table 3).

Table 3 Clinical Changes and implication

Discussion

This study outlines an institutional experience applying cell-free next generation sequencing from blood samples to the care of hospitalized children over an 8-month study period. We describe the clinical application of results from 142 cfNGS tests in the management of 110 subjects over 111 individual hospitalizations. In comparison to conventional testing as a reference standard, cfNGS determined to have a PPA of 89.6% and NPA of 52.3%. For 92 of the 142 cfNGS tests (64.8%) in which a putative pathogenic organism was determined to be clinically relevant, the relevant organisms were not identified by conventional testing. Furthermore, 32.4% of cfNGS results were directly applied to make a clinical change in management. This study illustrates an increased diagnostic yield for the detection of a putative pathogenic organism in subjects with cfNGS in comparison to conventional testing. Similar to previous studies, cfNGS had a higher rate of positivity for immunocompromised subjects, which is best explained by an increased pre-test probability of infection in these subjects [5, 10,11,12].

In the acute critical care setting, rapid diagnosis and treatment can improve patient outcomes. cfNGS offers the ability to test for many organisms at once in a broad metagenomic approach with a rapid turnaround time [1, 2, 7]. In a recent study of 350 septic adults, cfNGS demonstrated an increased yield in pathogen detection by approximately 34% over standard of care testing with an average return of result in 29 h [1]. With a rapid turnaround time and a broad range of pathogens detected by this technology, cfNGS may be a useful diagnostic tool in evaluating acutely ill patients, especially where expected diagnostic yield of conventional testing is expected to be low [4, 10, 13]. In our cohort, some cfNGS tests were specifically obtained to trend clearance of an infection or to support an alternative non-infectious diagnosis by obtaining a negative result. We furthermore suspect that the sensitivity and positive predictive value of cfNGS varies based on the clinical context in which testing is sent. For instance, cfNGS tests sent during cardiac admissions for post-operative fevers seem more likely to be negative than in an immunosuppressed subject admitted with sepsis.

Our results demonstrate both a higher degree of clinical utility and positivity of cfNGS than previously reported. In a recent study of the clinical utility of 82 cfNGS tests from mixed adults and pediatric subjects, positivity was found in 50 of 82 samples (61.0%), but with a clinical impact in just 6 (7.3%) instances [9]. This is in contrast to a single center retrospective review of 100 cfNGS results in pediatric subjects, for whom the study authors estimated sensitivity and specificity for diagnosis of a clinically relevant infection was 92 and 64% in a cohort heavily skewed towards oncology patients [6]. There are few studies that have validated cfNGS for specific disease etiologies, and in these studies, understanding results in relation to a historic reference standard as well as what constitutes a commensal organism has been challenging [14, 15]. The real-life interpretation of a test result is therefore ultimately left to the discretion of the treating clinicians, which in the case of this study was always done in collaboration with the infectious disease service.

Interestingly, treating physicians found the test to be useful even if it did not directly change clinical management. Perhaps, confirming standard of care testing may provide a sense of confidence when diagnosis and management decisions remain uncertain. This may provide an unmeasurable type of clinical utility to a provider. Treating a known infection may allow providers a chance to resist the urge to change antimicrobial coverage, especially in severely ill patients with prolonged and complicated infectious courses. In turn, this may mitigate the long-term emergence of resistant organisms.

Study limitations

This study evaluated cfNGS tests sent in hospitalized children at a single center with limited sample size. Given the retrospective nature of our analysis, the categorization of test indication and determination of clinical relevance of the test are subject to the recall bias of providers. In cases where anti-microbial changes or other changes in management were made, we did not collect data on long term follow up to determine the success of therapy. As is the case with previous studies evaluating cfNGS, the interpretation of the results as a representation of a true infection is also inherently subjective. When cfNGS results with numerous organisms, it may be more difficult to determine an organism’s likelihood of causation of symptoms, especially with organisms that may be considered commensal or otherwise part of normal flora. As cfNGS technology is evolving, however, quantitative values of organism DNA are now available and may help delineate likelihood of pathogenicity. The current available version of this test allows for only detection of organisms through DNA, so RNA based viruses such as SARS-CoV-2 would not be detected using current technology. This limitation could be addressed by also examining RNA. The benefit of hypothesis free testing can be clearly seen when a shortage of specific tests is made apparent. Finally, a standardized approach to guide test indication for both cfNGS and conventional testing does not exist at our institution, allowing for practice variation to potentially confound our results and limit generalizability. Furthermore, given the heterogeneity in the standard care evaluation of these subjects, a systematic comparison of cfNGS to a standardized reference was not possible, further limiting the interpretation and generalizability of our sensitivity analysis. We additionally adjudicated conventional testing as an aggregate of all non-cfNGS tests sent, which, given inherent differences in sensitivity and specificity of each available test, further adds unintended variability in the comparison between cfNGS and conventional testing.

Conclusion

In a retrospective review of the results of 142 cfNGS tests, we demonstrate the clinically utility in the management of acutely ill children. Future studies, both retrospective and prospective, are needed to clarify the optimal indications for testing, especially in relation to timing of testing within a hospitalization.

Availability of data and materials

The datasets generated and/or analyzed during the current study are not publicly available due inclusion of protected health information, but are available from the corresponding author on reasonable request. Sequencing was obtained via the Karius (Redwood City, CA) cfNGS commercial assay, and the sequencing data is therefore not available for public access due to proprietary concerns.

Abbreviations

cfNGS:

Cell-free plasma next-generation sequencing

RCHSD:

Rady children’s hospital San Diego

PPA:

Positive percent agreement

NPA:

Negative percent agreement

References

  1. 1.

    Blauwkamp TA, Thair S, Rosen MJ, Blair L, Lindner MS, Vilfan ID, et al. Analytical and clinical validation of a microbial cell-free DNA sequencing test for infectious disease. Nat Microbiol. 2019;4(4):663–74. https://doi.org/10.1038/s41564-018-0349-6.

    CAS  Article  PubMed  Google Scholar 

  2. 2.

    Hong DK, Blauwkamp TA, Kertesz M, Bercovici S, Truong C, Banaei N. Liquid biopsy for infectious diseases: sequencing of cell-free plasma to detect pathogen DNA in patients with invasive fungal disease. Diagn Microbiol Infect Dis. 2018;92(3):210–3. https://doi.org/10.1016/j.diagmicrobio.2018.06.009.

    CAS  Article  PubMed  Google Scholar 

  3. 3.

    Gu W, Miller S, Chiu CY. Clinical metagenomic next-generation sequencing for pathogen detection. Annu Rev Pathol. 2019;14:319–38.

    CAS  Article  Google Scholar 

  4. 4.

    Farnaes L, Wilke J, Ryan Loker K, Bradley JS, Cannavino CR, Hong DK, et al. Community-acquired pneumonia in children: cell-free plasma sequencing for diagnosis and management. Diagn Microbiol Infect Dis. 2019;94(2):188–91. https://doi.org/10.1016/j.diagmicrobio.2018.12.016.

    Article  PubMed  PubMed Central  Google Scholar 

  5. 5.

    Armstrong AE, Rossoff J, Hollemon D, Hong DK, Muller WJ, Chaudhury S. Cell-free DNA next-generation sequencing successfully detects infectious pathogens in pediatric oncology and hematopoietic stem cell transplant patients at risk for invasive fungal disease. Pediatr Blood Cancer. 2019;66(7):e27734. https://doi.org/10.1002/pbc.27734.

    CAS  Article  PubMed  Google Scholar 

  6. 6.

    Rossoff J, Chaudhury S, Soneji M, Patel SJ, Kwon S, Armstrong A, et al. Noninvasive Diagnosis of Infection Using Plasma Next-Generation Sequencing: A Single-Center Experience. In: Open forum infectious diseases. US: Oxford University Press; 2019. p. ofz327.

    Google Scholar 

  7. 7.

    Grumaz S, Stevens P, Grumaz C, Decker SO, Weigand MA, Hofer S, et al. Next-generation sequencing diagnostics of bacteremia in septic patients. Genome Med. 2016;8(1):73. https://doi.org/10.1186/s13073-016-0326-8.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  8. 8.

    Long Y, Zhang Y, Gong Y, Sun R, Su L, Lin X, et al. Diagnosis of sepsis with cell-free DNA by next-generation sequencing technology in ICU patients. Arch Med Res. 2016;47(5):365–71. https://doi.org/10.1016/j.arcmed.2016.08.004.

    CAS  Article  PubMed  Google Scholar 

  9. 9.

    Hogan CA, Yang S, Garner OB, Green DA, Gomez CA, Dien Bard J, et al. Clinical impact of metagenomic next-generation sequencing of plasma cell-free DNA for the diagnosis of infectious diseases: a multicenter retrospective cohort study. Clin Infect Dis. 2021;72(2):239–45.

  10. 10.

    Goggin KP, Gonzalez-Pena V, Inaba Y, Allison KJ, Hong DK, Ahmed AA, et al. Evaluation of plasma microbial cell-free DNA sequencing to predict bloodstream infection in pediatric patients with relapsed or refractory cancer. JAMA Oncol. 2020;6(4):552–6.

  11. 11.

    Fung M, Zompi S, Seng H, Hollemon D, Parham A, Hong DK, et al. Plasma cell–Free DNA Next-Generation Sequencing to Diagnose and monitor infections in allogeneic hematopoietic stem cell transplant patients. Open Forum Infect Dis. 2018;5(12):ofy301.

  12. 12.

    Zhou Y, Hemmige V, Dalai SC, Hong DK, Muldrew K, Mohajer MA. Utility of whole-genome next-generation sequencing of plasma in identifying opportunistic infections in HIV/AIDS. Open AIDS J. 2019;13(1):7–11.

  13. 13.

    Ramchandar N, Pong A, Anderson E. Identification of disseminated toxoplasmosis by plasma next-generation sequencing in a teenager with rapidly progressive multiorgan failure following haploidentical stem cell transplantation. Pediatric Blood Cancer. 2020;67(4):e28205.

  14. 14.

    Niles DT, Wijetunge DSS, Palazzi DL, Singh IR, Revell PA. Plasma metagenomic next-generation sequencing assay for identifying pathogens: a retrospective review of test utilization in a large children’s hospital. J Clin Microbiol. 2020;58(11) [cited 2021 Mar 1] Available from: https://jcm.asm.org/content/58/11/e00794-20.

  15. 15.

    Lee RA, Dhaheri FA, Pollock NR, Sharma TS. Assessment of the clinical utility of plasma metagenomic next-generation sequencing in a pediatric hospital population. J Clin Microbiol. 2020;58(7) [cited 2021 Mar 1] Available from: https://jcm.asm.org/content/58/7/e00419-20.

Download references

Funding

This study was not funded. Of note, cell free plasma NGS was obtained commercially through Karius (Redwood City, CA). Karius did not fund this study and was not involved with manuscript preparation.

Author information

Affiliations

Authors

Contributions

Authors LF, NC, and JW conceived the initial study design. Authors JW and JF designed the data collection tools. Authors NR, LF, JW, CC, AP, AT, LP, HH, and JF revised the study design and data collection tools and collected data and performed data analysis. Authors JW and NR wrote the initial draft of the manuscript. Authors LF, NC, JW, JF, CC, AP, AT, LP, and HH edited and revised the manuscript. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Nicole G. Coufal.

Ethics declarations

Ethics approval and consent to participate

The Institutional Review Board of Rady Children’s Hospital San Diego (RCHSD) and the Human Research Protections Program at the University of California San Diego approved this study. All methods were performed in accordance with the relevant guidelines and regulations. A waiver of informed consent was granted by the IRB of Rady Children’s Hospital San Diego (RCHSD) and the Human Research Protections Program at the University of California San Diego for this retrospective study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Supplemental Table 1

. Conventional Test Results. Supplemental Table 2. Conventional Testing vs cfNGS Results. Supplemental Table 3. PPA and NPA calculations.

Additional file 2.

 Full conventional and cfNGS results with adjudication.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wilke, J., Ramchandar, N., Cannavino, C. et al. Clinical application of cell-free next-generation sequencing for infectious diseases at a tertiary children’s hospital. BMC Infect Dis 21, 552 (2021). https://doi.org/10.1186/s12879-021-06292-4

Download citation

Keywords

  • Metagenomics
  • Cell-free plasma
  • Next-generation sequencing
  • Children