- Research article
- Open access
- Published:
Performance of SARS-CoV-2 antigen testing in symptomatic and asymptomatic adults: a single-center evaluation
BMC Infectious Diseases volume 21, Article number: 1071 (2021)
Abstract
Background
Antigen testing offers rapid and inexpensive testing for SARS-CoV-2 but concerns regarding performance, especially sensitivity, remain. Limited data exists for use of antigen testing in asymptomatic patients; thus, performance and reliability of antigen testing remains unclear.
Methods
148 symptomatic and 144 asymptomatic adults were included. A nasal swab was collected for testing by Quidel Sofia SARS IFA (Sofia) as point of care. A nasopharyngeal swab was also collected and transported to the laboratory for testing by Cepheid Xpert Xpress SARS-CoV-2/Flu/RSV RT-PCR (Cepheid).
Results
Overall, Sofia had good agreement with Cepheid (> 95%) in adults, however was less sensitive. Sofia had a sensitivity of 87.8% and 33.3% for symptomatic and asymptomatic patients, respectively. Among symptomatic patients, testing > 5 days post symptom onset resulted in lower sensitivity (82%) when compared with testing within 5 days of symptom onset (90%). Of the four Sofia false-negative results in the asymptomatic cohort, 50% went on to develop COVID-19 disease within 5 days of testing. Specificity in both symptomatic and asymptomatic cohorts was 100%.
Conclusions
Sofia has acceptable performance in symptomatic adults when tested < 5 days of symptom onset. Caution should be taken when testing patients with ≥ 5 days of symptoms. The combination of low prevalence and reduced sensitivity results in relatively poor performance of in asymptomatic patients. NAAT-based diagnostic assays should be considered in when antigen testing is unreliable, particularly in symptomatic patients with > 5 days of symptom onset and asymptomatic patients.
Background
Throughout the SARS-CoV-2 pandemic, there has been a consistent need for rapid and accurate testing. Nucleic acid amplification test (NAAT)-based diagnostics, such as RT-PCR, remain the gold standard. In efforts to increase test capacity, antigen testing has been introduced as an alternative testing method. Currently, numerous antigen devices have been authorized for use by the FDA under Emergency Use Authorization (EUA) for testing in symptomatic patients. Advantages of antigen testing are certainly desirable- fast, less costly and near patient- and have been implemented in a variety of institutions, from universities to nursing homes. However, it is not without limitations, with the most significant limitation being reduced sensitivity as compared to NAAT-based assays. False-negative results can have compounding negative effects in terms of controlling and reducing community spread of SARS-CoV-2.
Infection with SARS-CoV-2 results in a variety of clinical presentations from no symptoms to fulminate pneumonia, where asymptomatic patients with SARS-CoV-2 infection often go undiagnosed or unrecognized, though the role these patients play in the spread of disease is still in question [1, 2]. Pre-symptomatic patients present their own unique set of challenges. Diagnosis typically occurs after symptom onset where the opportunity of viral transmission and spread may have already occurred [3,4,5]. Therefore, accurate detection of SARS-CoV-2 in pre-symptomatic individuals is of vital importance to controlling spread of COVID-19. Even though antigen testing is being widely used in a variety of settings, its performance remains understudied in a- and pre-symptomatic patients. Studies have shown reduced sensitivity of antigen testing in both symptomatic and asymptomatic populations but particularly among asymptomatic patients, with some reports as low as 40% even among those with known household contacts [6,7,8]. Sensitivity is further reduced when testing asymptomatic patients without known contacts [8], raising the question if antigen testing is a reliable method for use in this population.
Herein, we report the performance of the Quidel Sofia SARS Antigen FIA in a point of care setting for symptomatic and asymptomatic adult patients given the lack of performance data for this specific antigen test.
Methods
Ethics
All RT-PCR testing was performed as a part of routine clinical care, according to CLIA’88 regulations. This project was approved by the UPMC Quality Assurance Board and undertaken as a Quality Improvement Initiative and as such was not formally reviewed by the University of Pittsburgh Institutional Review Board. Informed verbal or written patient consent was waived as per the UPMC Quality Assurance Board. Patient data collected for this project were de-identified, obtained and stored securely in a REDCap database (NIH/NCATS UL1 TR000445) via HIPAA-compliant procedures. No administrative permission was required to access raw data for the study as approved by the UPMC Quality Assurance Board.
Patient selection and sample collection
148 symptomatic and 144 asymptomatic adult patients, ≥ 18 years of age, presenting to outpatient care at either a UPMC outpatient COVID-19 testing facility or the University of Pittsburgh Health Services center in Pittsburgh, PA were included in the study. Patients that were < 18 years of age were excluded. Patients were classified as symptomatic or asymptomatic as per the physician order or chart review. Patients were considered symptomatic if they presented with any of the following symptoms: fever, cough, congestion, shortness of breath, headache, loss of taste/smell, sore throat, abdominal pain, vomiting, nausea, diarrhea, myalgia, fatigue and chills. Patients were classified as asymptomatic if they were without symptoms at time of testing. Testing was performed due to known exposure or because it was required for work, travel/re-entry, pre-procedural or other asymptomatic screening.
Two samples were collected from each patient included in the study. The first specimen, as per standard of care (SOC), was a nasopharyngeal (NP) swab that was collected and placed in viral transport media (VTM) (Remel M4-RT, Remel, San Diego, CA). NP samples were transported to the clinical microbiology laboratory for testing by SOC RT-PCR. A second specimen, nasal swab, was collected either by the healthcare provider (COVID testing site) or self-collected under the supervision of a healthcare provider (student health) for antigen testing. Direct antigen testing was performed at the site of collection (eg, testing center or student health). Testing was performed from January 2021 until March 2021.
Antigen testing
The Quidel Sofia SARS IFA antigen assay (Sofia; San Diego, CA) is authorized by the FDA for qualitative detection of SARS-CoV-2 by detecting the presence of SARS-CoV-2 nucleocapsid antigen by a sandwich immunoassay methodology. Sofia was chosen for the study due to lack of performance data for this assay and easy access to Sofia instrument and tests. Sofia was performed directly on nasal swabs as per the manufacturer’s instructions [9]. Testing was performed as a point of care test at the site of collection by trained personnel as per CLIA’ 88 for waived laboratory testing. Collection site was a sample collection center for patients requiring testing for SARS-CoV-2.
Nucleic acid testing
SOC PCR testing on NP swabs in VTM were performed on the Cepheid GeneXpert® Xpress SARS-CoV-2/Flu/RSV real-time RT-PCR assay (Cepheid; Sunnyvale, CA) [10], which is authorized by FDA for the qualitative detection of SARS-CoV-2. Sample was transported to the clinical laboratory and testing was performed on-site at the clinical laboratory as per the manufacturer’s instructions and according to CLIA’88 for moderate-high complexity testing. Testing was performed within 48 h of sample receipt. Quality control was performed as per the laboratory’s individual quality control plan and according to CLIA’88 requirements. Cepheid was the reference method for the study.
Data collection and analysis
Patients were stratified into symptomatic or asymptomatic groups for data analysis. Review of the electronic medical record (EMR) was performed to confirm proper designation of specimens as symptomatic or asymptomatic and ensure exclusion of repeat specimens. Patient demographics, reason for testing and exposure history were collected. A positive exposure was defined as greater than 15 min of contact, without a mask, with an individual with a documented positive SARS-CoV-2 result. Twenty-four patients categorized as asymptomatic surveillance were excluded from clinical analysis as no medical note could be identified during the review of the electronic medical record. For symptomatic patients, the days from symptom onset until testing was also collected and grouped into < 5 days and ≥ 5 days. Performance of Sofia was determined by comparing to the results from Cepheid. Percent agreement, sensitivity, specificity, positive and negative predictive values (PPV and NPV) as well as overall prevalence were calculated using contingency tables. Prevalence was calculated based on Cepheid positivity as the gold standard. Positive and negative predictive values, factoring in prevalence for each cohort, were calculated as follows:
For asymptomatic patients, discrepancies between Sofia and Cepheid were clinically adjudicated by in-depth chart review. Statistics including averages, range and Student’s T-Test was determined using Microsoft Excel. T-test was used for comparison of CT values; p < 0.05 were considered statically significant.
Statistical considerations
For assessing diagnostic tests, the general recommendation is to include 200–300 patients for appropriate power to observe statistical significance [11]. However, prevalence is one factor in calculating the minimum sample size required, along with assay sensitivity and specificity. Based on the observed prevalence, our study is sufficiently powered for the symptomatic cohort but may be underpowered for the asymptomatic cohort in determining antigen test performance (eg, sensitivity and specificity).
Results
During the study period, the prevalence of COVID-19 in our cohorts, as determined by Cepheid positivity, was 27.7% for symptomatic patients and 4.2% for asymptomatic patients. In general, the Sofia performed well with an overall percent agreement of 96.6% for symptomatic patients and 97.2% for asymptomatic patients (Table 1A and B). For symptomatic patients, overall sensitivity was 87.8%, with five false-negative (FN) results by Sofia (12.2% FN). Of the five FN, the cycle threshold (CT), which is often used as a loose semi-quantitation of sample viral load, ranged from 22.1 to 32.1 and spans what is generally considered to be strongly to moderately positive. For all Cepheid positives in the symptomatic cohort, the average CT was 23.1 (range of 14.2–33.5; Fig. 1). Comparatively, the average CT the Sofia FN was 27.7, which barely missed statistical significance (p = 0.05; data not shown). The NPV and PPV, when taking into account prevalence in our symptomatic population, was 95.5% and 100%, respectively. When Sofia performance was stratified by number of days from symptom onset to testing (< 5 or ≥ 5 days), sensitivity was better when testing at < 5 days of symptom onset (90% for < 5 days vs 81.8% for ≥ 5 days), thus resulting in a slight reduction in NPV for those tested at ≥ 5 days of symptom onset (Table 1A).
For asymptomatic patients, Sofia’s overall sensitivity was 33.3%, with 4 FN results (66.7% FN; Table 1B). Of the 4 FN, the average CT was 34.6 with a range of 29.5–37.4 (Fig. 1), with 3 of the 4 FN demonstrating CT > 33, which is considered weakly positive. For all Cepheid positives in the asymptomatic cohort, the average CT was 30.2 with a range of 15.4–37.4 and was statistically different from the average FN CT value (p = 0.002; Fig. 1). The NPV and PPV in the asymptomatic cohort was determined to be 97.2% and 100%, respectively, when accounting for a prevalence of 4.2%. No false-positives were observed during our study for either population, resulting in a specificity of 100% (Table 1).
Average age of symptomatic patients tested was 44 years. Of the symptoms recorded, just above half of the patients reported cough and congestion, with headache, sore throat, myalgia and fatigue being the next most common symptoms (Table 2A). Interestingly, fever was only reported in 19% of patients. Overall, patients reported having a total of 3–4 symptoms. Twenty and nine tenths percent (20.9%) of patients had a known exposure and on average, testing was performed at day 5 post-exposure (Table 2A). However, the majority of patients were tested after 3–4 days of symptoms (Table 2A). Average age of asymptomatic patients tested was 38 years. The most common reason, if recorded, for asymptomatic testing was other asymptomatic surveillance, with pre-surgical procedure screening being second. Interestingly, testing of asymptomatic individuals with a known exposure only represented 5.6% of our population (Table 2B). However, those who had a known exposure received testing on average at day 2–3 post-exposure as opposed to day 5, as seen in our symptomatic cohort (Table 2).
Of the four Sofia FN observed in the asymptomatic population, additional chart review was performed for clinical resolution. One patient did not have an encounter note near the test date in our EMR system and was therefore excluded from clinical assessment. One patient required testing for work, was asymptomatic and had no known prior COVID-19 disease. The CT value for this patient’s sample was > 35, where antigen testing has high likelihood to be falsely-negative [12]. The patient had a repeat COVID-19 test at an outside facility approximately 1 month later for unknown reasons, which was negative. From this review, it remains unclear whether this weak positive represents sporadic detection of unrecognized previous COVID-19 disease, asymptomatic carriage or an analytical false-positive. Interestingly, two of the patients that were FN by Sofia (50%) had known exposures, no prior known COVID-19 disease and went on to develop symptoms and COVID-19 disease around 5 days after initial testing. CT values for these two samples were > 35. These two cases likely represent detection of early SARS-CoV-2 infection and thus true positives by Cepheid.
Discussion
Overall, performance of Sofia in both patient populations was good, with > 95% agreement with Cepheid. Despite reports in the literature and media of high false-positive results when using antigen testing, we did not observe any false-positives during our study. However, this may be due to the limited number of asymptomatic patients in our study, resulting in the inability of our data to accurately reflect the specificity of the antigen test. Sofia sensitivity was less than Cepheid, which was not unexpected, missing nine true positives; five in the symptomatic cohort and four in the asymptomatic cohort. Additionally, for symptomatic patients, sensitivity of Sofia further decreased when testing patients with ≥ 5 days from symptom onset. Given that the median time to symptom onset is approximately 5 days and is near the average time that patients receive testing, the lack of sensitivity of Sofia at this time point may result in higher false-negative rates for this subset of patients, especially if testing is delayed by one or two days [13]. Sofia’s reduced sensitivity at > 5 days is not surprising given this is a known limitation of the assay, which is stated in the Sofia package insert. However, our data further strengthens and highlights this important consideration, as this reduction in the performance of antigen tests is often overlooked in clinical practice and rarely mentioned in testing guidelines or recommendations. Efforts should be taken to better educate those using antigen testing of this limitation, with careful consideration when using Sofia, such that NAAT testing be performed initially or to confirm antigen negative results on patients who present ≥ 5 days from symptom onset.
Concerns have been raised regarding the use of antigen testing in asymptomatic patients. Our study provides additional insight into antigen test performance in this population, where the pre-test probability is low. Sofia has poor sensitivity (33.3%) when testing this population, which is similar to what is described in the literature to date. While our NPV was high, it is important to consider that NPV is highly dependent on the prevalence in the intended population and thus may differ for other institutions. Nevertheless, lower prevalence is expected in an asymptomatic population, as compared to symptomatic patients, and thus our data likely represents real-world test performance. Interestingly, 2 of the 4, were patients who had a known exposure and went on to develop COVID-19 disease within 5 days from testing. This suggests that these patients were indeed true positives that were detected by Cepheid but missed by Sofia. The Cepheid CT values for these two samples were both > 35, where antigen testing is inadequate. Taking the CT and clinical context together, this suggests that these patients were early in their infection and represent the pre-symptomatic patient population, which remain a crux in controlling further transmission and spread of SARS-CoV-2. Reliable results are paramount to aid in the rapid implementation of public health measures to help reduce spread of the virus in the community. Thus, for accurate detection of SARS-CoV-2 in potentially pre-symptomatic patients, with known exposures, our data suggests that testing should be NAAT-based. Additionally, these two examples should also give pause to whether the strength of CT values can be associated with infectious or non-infectious states. CT values alone cannot predict contagiousness and the results must be taken in context with the clinical picture, such as exposure date and history, co-morbidities and symptomology.
Taken together, antigen testing can help expand rapid and affordable SARS-CoV-2 testing. Most importantly, careful consideration should be taken as to when and on whom the test is used to ensure that results are reliable. For certain patient populations, such as asymptomatic patients or those who have been symptomatic for 35 days, NAAT-based diagnostics have a clear advantage and should be strongly recommended for primary use or incorporated into well-designed reflex algorithms to ensure antigen results can be trusted and are actionable.
Conclusions
Sofia has acceptable performance in symptomatic adults when testing is performed < 5 days of symptoms. Low prevalence and reduced sensitivity results in relatively poor performance in asymptomatic patients. NAAT-based diagnostic assays and practical reflex algorithms should be considered in when antigen testing is unreliable, particularly in symptomatic patients with > 5 days of symptom onset and asymptomatic patients.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request. Request can be made to Dr. Stephanie L. Mitchell; slm198@pitt.edu.
Abbreviations
- IFA:
-
Immunofluorescence assay
- RT-PCR:
-
Reverse transcriptase polymerase chain reaction
- NAAT:
-
Nucleic acid amplification test
- EUA:
-
Emergency Use Authorization
- SOC:
-
Standard of care
- NP:
-
Nasopharyngeal
- VTM:
-
Viral transport media
- PPV:
-
Positive predictive value
- NPV:
-
Negative predictive values
- EMR:
-
Electronic medical record
- FN:
-
False-negative
- CT :
-
Cycle threshold
References
Rothe C, Schunk M, Sothmann P, Bretzel G, Froeschl G, Wallrauch C, Zimmer T, Thiel V, Janke C, Guggemos W, Seilmaier M, Drosten C, Vollmar P, Zwirglmaier K, Zange S, Wolfel R, Hoelscher M. Transmission of 2019-nCoV infection from an asymptomatic contact in Germany. N Engl J Med. 2020;382(10):970–1.
Bai Y, Yao L, Wei T, Tian F, Jin DY, Chen L, Wang M. Presumed asymptomatic carrier transmission of COVID-19. JAMA. 2020;323(14):1406–7.
Buitrago-Garcia D, Egli-Gany D, Counotte MJ, Hossmann S, Imeri H, Ipekci AM, Salanti G, Low N. Occurrence and transmission potential of asymptomatic and presymptomatic SARS-CoV-2 infections: a living systematic review and meta-analysis. PLoS medicine. 2020;17(9):e1003346.
Arons MM, Hatfield KM, Reddy SC, Kimball A, James A, Jacobs JR, Taylor J, Spicer K, Bardossy AC, Oakley LP, Tanwar S, Dyal JW, Harney J, Chisty Z, Bell JM, Methner M, Paul P, Carlson CM, McLaughlin HP, Thornburg N, Tong S, Tamin A, Tao Y, Uehara A, Harcourt J, Clark S, Brostrom-Smith C, Page LC, Kay M, Lewis J, Montgomery P, Stone ND, Clark TA, Honein MA, Duchin JS, Jernigan JA, Public H-S, King C, Team CC-I (2020) Presymptomatic SARS-CoV-2 Infections and Transmission in a Skilled Nursing Facility. The New England journal of medicine 382 (22):2081-2090.
Meyerowitz EA, Richterman A, Bogoch, II, Low N, Cevik M (2020) Towards an accurate and systematic characterisation of persistently asymptomatic infection with SARS-CoV-2. Lancet Infect Dis.
Linares M, Perez-Tanoira R, Carrero A, Romanyk J, Perez-Garcia F, Gomez-Herruz P, Arroyo T, Cuadros J. Panbio antigen rapid test is reliable to diagnose SARS-CoV-2 infection in the first 7 days after the onset of symptoms. J Clin Virol. 2020;133:104659.
Pray IW, Ford L, Cole D, Lee C, Bigouette JP, Abedi GR, Bushman D, Delahoy MJ, Currie D, Cherney B, Kirby M, Fajardo G, Caudill M, Langolf K, Kahrs J, Kelly P, Pitts C, Lim A, Aulik N, Tamin A, Harcourt JL, Queen K, Zhang J, Whitaker B, Browne H, Medrzycki M, Shewmaker P, Folster J, Bankamp B, Bowen MD, Thornburg NJ, Goffard K, Limbago B, Bateman A, Tate JE, Gieryn D, Kirking HL, Westergaard R, Killerby M, Group CC-SL. Performance of an antigen-based test for asymptomatic and symptomatic SARS-CoV-2 testing at two university campuses - wisconsin, september-october 2020. MMWR Morb Mortal Wkly Rep. 2021;69(5152):1642–7.
Albert E, Torres I, Bueno F, Huntley D, Molla E, Fernandez-Fuentes MA, Martinez M, Poujois S, Forque L, Valdivia A, Solano de la Asuncion C, Ferrer J, Colomina J, Navarro D. Field evaluation of a rapid antigen test (Panbio COVID-19 Ag Rapid Test Device) for COVID-19 diagnosis in primary healthcare centres. Clin Microbiol Infect. 2021;27(3):472.
Quidel (2020) Quidel Sofia SARS Antigen IFA https://www.fda.gov/media/137885/download
Cepheid (2021) Xpert Xpress SARS-CoV-2/Flu/RSV. https://www.fda.gov/media/142437/download
Bujang MA, Adnan TH. Requirements for minimum sample size for sensitivity and specificity analysis. J Clin Diagn Res. 2016;10(10):YE01–6.
Gomez Marti JL, Gribschaw J, McCullough M, Mallon A, Acero J, Kinzler A, Godesky J, Heidenrich K, Iagnemma J, Pasculle AW, Phan T, Hoberman A, Williams JV, Mitchell SL, Wells A. Differences in detected viral loads guide use of SARS-CoV-2 antigen-detection assays towards symptomatic college students and children. MedRxiv. 2021. https://doi.org/10.1101/2021.01.28.21250365.
Lauer SA, Grantz KH, Bi Q, Jones FK, Zheng Q, Meredith HR, Azman AS, Reich NG, Lessler J. The incubation period of coronavirus disease 2019 (COVID-19) from publicly reported confirmed cases: estimation and application. Ann Intern Med. 2020;172(9):577–82.
Acknowledgements
We would like to thank UPMC SouthSide Collection Center, University of Pittsburgh Student Health Services, UPMC Clinical Microbiology Laboratory for support of this study.
Funding
Financial support was provided by Cepheid and also provided input into study design. Cepheid had no role in the collection, analysis, and interpretation of data or in writing the manuscript.
Author information
Authors and Affiliations
Contributions
SLM developed the study design, execution, oversight, data analysis and the lead writer of the manuscript. SO collected patient samples and performed antigen testing. TF performed retrospective chart review and data analysis and a contributor to manuscript edits. MCF performed retrospective chart review and data analysis and a contributor to manuscript edits. MA1 collected patient samples and performed antigen testing. MA2 collected patient samples and performed antigen testing. JG provided administrative support and laboratory testing oversight. JS providing administrative support and collected patient samples. AH assisted in study design, provided instrumentation for antigen testing and a contributor to manuscript edits. AW assisted in administrative support, study design and execution and a contributor to manuscript edits. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
All RT-PCR testing was performed as a part of routine clinical care, according to CLIA’88 regulations. This project was approved by the UPMC Quality Assurance Board and undertaken as a Quality Improvement Initiative and as such was not formally reviewed by the University of Pittsburgh Institutional Review Board. Informed verbal or written patient consent was waived as per the UPMC Quality Assurance Board. Patient data collected for this project were de-identified, obtained and stored securely in a REDCap database (NIH/NCATS UL1 TR000445) via HIPAA-compliant procedures. No administrative permission was required to access raw data for the study as approved by the UPMC Quality Assurance Board.
Consent for publication
Not applicable.
Competing interests
Dr. Mitchell reports grants and non-financial support from Cepheid for conducting this quality improvement project; and after completion of this work, Dr. Mitchell is now employed by Cepheid.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Mitchell, S.L., Orris, S., Freeman, T. et al. Performance of SARS-CoV-2 antigen testing in symptomatic and asymptomatic adults: a single-center evaluation. BMC Infect Dis 21, 1071 (2021). https://doi.org/10.1186/s12879-021-06716-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12879-021-06716-1