Skip to main content

Test concordance and diagnostic accuracy of three serological assays for detection of anti-SARS-CoV-2 antibody: result from a population-based sero-epidemiological study in Delhi

Abstract

Background

Several methodological tests are available to detect SARS-CoV-2 antibody. Tests are mostly used in the aid of diagnosis or for serological assessment. No tests are fully confirmatory and have variable level of diagnostic ability. We aimed at assessing agreement with three serological tests: quantitative anti receptor binding domain ELISA (Q-RBD), qualitative ELISA (WANTAI SARS-CoV-2 Ab) and qualitative chemiluminescence assay (CLIA).

Methods

This study was a part of a large population based sero-epidemiological cohort study. Participants aged 1 year or older were included from 25 randomly selected clusters each in Delhi urban (urban resettlement colony of South Delhi district) and Delhi rural (villages in Faridabad district, Haryana). Three type of tests were applied to all the baseline blood samples. Result of the three tests were evaluated by estimating the total agreement and kappa value.

Results

Total 3491 blood samples collected from March to September, 2021, out of which 1700 (48.7%) from urban and 1791 (51.3%) from rural. Overall 44.1% of participants were male. The proportion of sero-positivity were 78.1%, 75.2% and 31.8% by Wantai, QRBD and CLIA tests respectively. The total agreement between Wantai and QRBD was 94.5%, 53.1% between Wantai and CLIA, and 56.8% between QRBD and CLIA. The kappa value between these three tests were 0.84 (95% CI 0.80–0.87), 0.22 (95% CI 0.19–0.24) and 0.26 (95% CI 0.23–0.28).

Conclusions

There was strong concordance between Wantai and QRBD test. Agreement between CLIA with other two tests was low. Wantai and QRBD tests measuring the antibody to same S protein can be used with high agreement based on the relevant scenario.

Peer Review reports

Background

Approach to test anti-SARS-CoV-2 antibody level is a valuable aspect of this COVID-19 pandemic response. This is helpful to understand the extent of past infection as well as level of immune response at individual level and for serological surveillance in a population [1, 2]. This can also be used as an aid of diagnosis for recent past infection and among the suspected COVID-19 patient having post-COVID illness [2, 3]. The antibody test should have high level of accuracy to capture wide group of population especially the asymptomatic group [4]. There are several type of serological tests that detect IgG and or IgM using different methodology e.g. enzyme linked immunosorbent assay (ELISA), chemiluminescence assay (CLIA), lateral flow immunoassay [5]. The target protein and the type of antibody captured by the test kit influences the proportion of positive samples in a group of population. Therefore these assays have variable level of sensitivity, specificity and concordance [6, 7]. Due to these variable accuracy, the interpretation of the serological tests result of different tests are challenging in the context of epidemiological as well as clinical scenario which necessitates requirement of further study [8]. The present sero-prevalence study had used quantitative anti receptor binding domain ELISA (Q-RBD), qualitative ELISA (WANTAI SARS-CoV-2 Ab) [9] and qualitative chemiluminescence assay (CLIA) [10]. We aimed to assess the test concordance of these test kits for the SARS-CoV-2 specific antibody detection.

Methods

Design and participants

This study was part of a multi-centric population-based, age- stratified prospective cohort study. The study population was from selected clusters of rural and urban area. Any participants with age more than equal to 1 year were included considering the practical difficulty of obtaining consent, blood sample among the infant age group. The participants not willing to give blood sample were excluded. In each area, all family members of ≥ 1 year age from the consecutive families were included from 25 randomly selected clusters assuming four participants per family. In rural area, individual village was considered as cluster whereas municipality ward was taken as cluster in urban area. We are reporting the baseline data only.

The data collection period was from March to September, 2021. Informed written consent was obtained from all the participants and the samples were collected maintaining all recommended precautions. The study was approved by the Institutional ethics committee of AIIMS, New Delhi.

Procedure and setting

Participants of both urban and rural area were included. The urban area of Delhi state was a resettlement colony in South Delhi district. The area was mainly populated migratory people and of lower economic strata having a population of 36,000 as of 2021. The rural participants were from the rural area of Faridabad district of Haryana state which comes under Delhi National Capital Region (NCR). The area was the field practice area of the investigating institution. It had total 28 villages with a population of 101,000 in 2021. Out of the 28 villages, 25 villages were selected by probability to proportion to size sampling method. The data collection team consists of a research officer, nursing staff, lab technician and a field attendant. They were directed to start recruiting the participants from a randomly selected lane of a site preferably from the centre of the cluster. All the participants of aged more than equal to 1 year were approached from the consecutive houses. The rule of left were adopted to move at the end of the lane. After taking due consent and or assent, 5 ml of venous blood sample was collected in clot activator vials (yellow top with gel separator) from each participant. The participants were interviewed by trained staff to collect information related to basic socio-demographic factor, vaccination details etc. The serum was separated within 2 h of collection. The serum samples were transported to laboratory on the same day maintaining the cold chain temperature of 2–8 degree Celsius. In the laboratory, the sera were stored at -80 degree temperature until analysed.

Outcome variables

Antibody level to SARS-CoV-2 virus in human serum was assessed quantitatively and qualitatively based on the type of test kit.

Laboratory tests

Each of the serum sample was analysed by three different serological tests within seven days of separation of serum. The first test was a qualitative assessment of SARS-CoV-2 antibody by ELISA method using Wantai SARS-CoV-2 Ab ELISA kit. The second assay was quantitative estimation of the SARS-CoV-2 antibody by in house developed ELISA kit. The third test was qualitative assay by chemiluminescence method using Abbott laboratories (Table 1).

Table 1 Specification details of the three serological tests used to detect Anti-SARS-CoV-2 antibodies

Methodology for WANTAI SARS‐CoV‐2 Antibody ELISA

This was done by sandwich ELISA kit using a polystyrene microwell in two steps. In the first step, the serum of the participants were added into which the antibody got adhered to the wells. The free antibody was then washed away. The second step was to add recombinant SARS-CoV-2 antigen which was conjugated with Horseradish Peroxidase (HRP-Conjugate) was added. The antigen got bound to the antibody which was captured in the first step in the well forming a “sandwich” immune complex. This reaction was reflected by the yellow colouration once a colourless chromogen solution came in contact with the immune complex. Those sample having the absorbance to Cut-off ratio of ≥ 1.0 was accepted as positive.

Methodology for QRBD

This was also done in a two-step approach. A 96 well polystyrene plate was coated with recombinant spike protein. The anti-RBD antibody in the participant’s serum reacted with the coated RBD antigen while extra unbound antibody was washed away. HRP-conjugated anti-IgG containing solution was added which bound to the human IgG bound to the RBD protein in the micro well. The reaction was captured by the blue to yellow colouration after adding Tetramethylbenzidine (TMB) and sulphuric acid (H2SO4). The colour intensity was measured by the absorbance of the solution by 450 nm of wave. The intensity of the colour was directly proportional to the amount of anti-RBD antibodies present in the serum.

Methodology for CLIA

This was also a two-step immunoassay with chemiluminescent micro particle technology principle. Micro particle coated with SARS-CoV-2 antigen were combined with assay diluent followed by incubation. The antibodies present in the participant’s serum binds with the antigen coated micro particle. Anti-human IgG labelled with acridinium conjugate was added followed by pre-trigger and trigger solution. The test reaction was measured by system optics and expressed as relative light unit (RLU). The level of RLU was directly proportional to the amount of IgG. It was then compared to the calibrator RLU to determine the presence and absence of IgG antibodies against SARS-CoV-2.

Data analysis

The data were extracted in Microsoft excel and analysed in STATA V12 statistical software. The qualitative test results of antibody were expressed as proportion whereas the quantitative level of antibody were expressed in mean value with 95% confidence interval. The test accuracy measure was estimated by sensitivity and specificity considering each test as the gold standard. Concordance were expressed by total agreement calculated between QRBD & CLIA, WANTAI & QRBD and WANTAI & CLIA and by Cohen’s kappa value for the agreement beyond chance.

Results

The data collection period was from March to September, 2021 in Delhi urban and in rural area. The total collected sample was 3491 where 1700 (48.7%) was from urban area and 1791 (51.3%) from rural area. In the urban area 9.3% participants were under 18 years of age whereas in rural area it was 18.5%. Overall 44.1% of the participants were male with a slight higher proportion in rural area (46.8%). In urban area 94.0% participants were un-vaccinated whereas in rural it was 84.7% (Table 2). A total of 78.1% samples were sero-positive by Wantai, 75.2% by QRBD and 31.8% by CLIA tests respectively (Table 3). As per the QRBD test, the mean antibody level among positive group was 270.6 BAU/ml (SD = 424.3, 95% CI 253.5–287.8) and among negative group it was 2.4 BAU/ml (SD = 2.1, 95% CI 2.3–2.6). The sensitivity and specificity of QRBD in respect to WANTAI was 94.6% (95% CI 93.6–95.4%) and 94.3% (95% CI 92.3–95.8%). The sensitivity and specificity of CLIA in respect to WANTAI was 40.3% (95% CI 38.4–42.2%) and 98.4% (95% CI 97.3–99.2%) respectively (Table 4). The sensitivity and specificity of CLIA in respect to QRBD was 42.9% (95% CI 41.0–44.9%) and 99.0% (95% CI 98.1–99.5%) (Table 5). The total agreement between Wantai and QRBD was 94.5%, 53.1% between Wantai and CLIA, and 56.8% between QRBD and CLIA. The kappa value between these three tests were 0.84 (95% CI 0.80–0.87), 0.22 (95% CI 0.19–0.24) and 0.26 (95% CI 0.23–0.28) (Tables 4 and 5). The total proportion of positive test result was found almost similar among the population both in vaccinated and unvaccinated. This finding was consistent across all the three tests. When compared the seropositivity among the recipients of single or two doses of vaccine, it was seen that the proportion of positive tests were higher among those getting two doses of vaccine (Table 6).

Table 2 Distribution of the participants according to age, sex and vaccination status in both rural and urban area
Table 3 Proportion of qualitative result of the serum antibody by WANTAI, QRBD and CLIA tests
Table 4 Diagnostic accuracy of QRBD and CLIA test kit in reference to WANTAI and level of agreement
Table 5 Diagnostic accuracy of CLIA test kit in reference to QRBD and level of agreement
Table 6 Serolo-positivity percentage based on the vaccination status as per the Wantai, QRBD and CLIA test

Discussion

We assessed the concordance of the three test kits in this population based seroepidemiological study. There was a strong agreement beyond chance between the WANTAI and the QRBD kit (kappa = 0.83) [7]. The total agreement was also high (92.3%) reflecting a high level of concordance.

The diagnostic accuracy of QRBD test in respect to Wantai test was high. Whereas for the CLIA it was low. There was high level of false negative result by CLIA which were positive by Wantai and QRBD. For the CLIA test kit, the total agreement and the kappa value was low in reference to both WANTAI (total agreement = 53.1%, kappa = 0.22) as well as QRBD (total agreement = 56.6%, kappa = 0.26). This can be due to the difference in the target protein of the CLIA and other two tests. The CLIA test captures the antibody against the nucleo-capsid protein (NP) of the virus which is used for the genomic packing while replication [11]. Whereas the other two tests captures the receptor binding domain of spike protein which is used for the binding with the host cell receptor and is a potential target of all the vaccines [12]. Though the WANTAI captures total antibody (IgA + IgG + IgM) and the QRBD captures the IgG only, strong agreement was found. This may be due to the higher proportion of IgG and the longer duration of IgG persists both in natural infection as well as in vaccinated individuals [13, 14]. There was structural and functional difference between the two protein as well as the expression in respect to the antibody production. Therefore the concordance of CLIA test with the other two tests were poor. In a study done by Liu et al. significantly low positive rate was found by tests detecting antibody against nucleocapsid protein [6]. There was inherited difference of the three tests regarding the ability to capture the type of antibody. In spite of this, the evidence of this study helped us to get a further understanding of prevalence obtained through these tests and the degree of agreement. Therefore, the sero-prevalence data always necessitates the understanding of the tests criteria for a clearer picture of the pandemic scenario.

Our study included samples from both urban and rural area. Larger sample size was another strength of this study. The selected age group was also wider having participants from very young to elderly participants.

We did not assess in respect to known serum sample from RTPCR confirmed participants. The urban resettlement area was purposively chosen. Moreover it didn’t represent the whole urban Delhi population.

Conclusion

The variable level of the diagnostic accuracy and agreement reflects the diagnostic capabilities of different serological assessment. Serological tests are used in clinical and epidemiological context [8, 9]. Therefore, choosing an appropriate test and interpretation of the serology data should be based on such evidences.

Availability of data and materials

The data and materials is not available publicly. It can be obtained upon reasonable request to the corresponding author.

Abbreviations

SARS-CoV-2:

Severe Acute Respiratory Syndrome Coronavirus-2

WHO:

World Health Organization

COVID:

Coronavirus Disease

ELISA:

Enzyme-linked Immunosorbent Assay

Q-RBD:

Quantitative-Receptor Binding Domain

CLIA:

Chemiluminescence Assay

AIIMS:

All India Institute of Medical Sciences

NCR:

Delhi National Capital Region

THSTI:

Translational Health Science and Technology Institute

OD:

Optical Density

ICMR:

Indian Council of Medical Research

References

  1. West R, Kobokovich A, Connell N, Gronvall GK. COVID-19 antibody tests: a valuable public health tool with limited relevance to individuals. Trends Microbiol. 2021;29(3):214–23.

    Article  CAS  Google Scholar 

  2. CDC. Labs [Internet]. Centers for Disease Control and Prevention. 2020. https://www.cdc.gov/coronavirus/2019-ncov/lab/resources/antibody-tests-guidelines.html. Accessed 19 Au 2022.

  3. Health C for D and R. In Vitro Diagnostics EUAs. FDA. 2022 Oct 6; https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/in-vitro-diagnostics-euas. Accessed 17 Aug 2022.

  4. Milani GP, Dioni L, Favero C, Cantone L, Macchi C, Delbue S, et al. Serological follow-up of SARS-CoV-2 asymptomatic subjects. Sci Rep. 2020;10(1):20048.

    Article  CAS  Google Scholar 

  5. Rastawicki W, Rokosz-Chudziak N. Characteristics and assessment of the usefulness of serological tests in the diagnostic of infections caused by coronavirus SARS-CoV-2 on the basis of available manufacturer’s data and literature review. Przegl Epidemiol. 2020;74(1):49–68.

    Article  Google Scholar 

  6. Liu W, Liu L, Kou G, Zheng Y, Ding Y, Ni W, et al. Evaluation of nucleocapsid and spike protein-based enzyme-linked immunosorbent assays for detecting antibodies against SARS-CoV-2. J Clin Microbiol. 2020;58(6):e00461.

    Article  Google Scholar 

  7. Guevara-Hoyer K, Fuentes-Antrás J, De la Fuente-Muñoz E, Rodríguez de la Peña A, Viñuela M, Cabello-Clotet N, et al. Serological tests in the detection of SARS-CoV-2 antibodies. Diagnostics. 2021;11(4):678.

    Article  CAS  Google Scholar 

  8. Ong DSY, Fragkou PC, Schweitzer VA, Chemaly RF, Moschopoulos CD, Skevaki C. How to interpret and use COVID-19 serology and immunology tests. Clin Microbiol Infect. 2021;27(7):981–6.

    Article  CAS  Google Scholar 

  9. Wantai : COVID-19 Serology and Molecular Tests. Wantai BioPharm. http://www.ystwt.cn/covid-19/. Accessed 25 Apr 2021.

  10. Abbott ARCHITECT SARS-CoV-2 IgG Instructions for Use. H14806R03. May 2020—Google Search. https://www.fda.gov/media/137383/download. Accessed 10 Jul 2021.

  11. Cubuk J, Alston JJ, Incicco JJ, Singh S, Stuchell-Brereton MD, Ward MD, et al. The SARS-CoV-2 nucleocapsid protein is dynamic, disordered, and phase separates with RNA. Nat Commun. 2021;12(1):1936.

    Article  CAS  Google Scholar 

  12. Huang Y, Yang C, Xu XF, Xu W, Liu SW. Structural and functional properties of SARS-CoV-2 spike protein: potential antivirus drug development for COVID-19. Acta Pharmacol Sin. 2020;41(9):1141–9.

    Article  Google Scholar 

  13. Zervou FN, Louie P, Stachel A, Zacharioudakis IM, Ortiz-Mendez Y, Thomas K, et al. SARS-CoV-2 antibodies: IgA correlates with severity of disease in early COVID-19 infection. J Med Virol. 2021;93(9):5409–15.

    Article  CAS  Google Scholar 

  14. Ma H, Zeng W, He H, Zhao D, Jiang D, Zhou P, et al. Serum IgA, IgM, and IgG responses in COVID-19. Cell Mol Immunol. 2020;17(7):773–5.

    Article  Google Scholar 

Download references

Acknowledgements

We thank the WHO Country Office, India team particularly Dr Mohammad Ahmad (National Professional Officer, WHO) and Dr. Anisur Rahman (Health Emergencies and Research Officer, WHO) for continuous support. Special thanks to the participants who allowed us to investigate the extent of infection, as determined by seropositivity in the general population, in which COVID-19 virus infection has been reported.

Funding

This work was supported by a research grant (Ref No: 2020/1085497, Purchase Order: 202630166) from the WHO Country Office, New Delhi 110016, India.

Author information

Authors and Affiliations

Authors

Contributions

The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. This manuscript represents the honest work done in this multisite sero-epidemiological study. Concept: PM, SK, RG, MA, SKR, AR, KY, MB, PH. Design: PM, SK, RG, MA, PKC, GRM, SKR, AR, MS, KY, MB, PH. Literature search: PM, SK, SupM, MS. Data acquisition and analysis: PM, SK, SupM, PKC, GRM, SunM, MS. Manuscript preparation: PM, SK, SupM, MS. Manuscript editing: PM, SK, MA, SupM, SunM, MS. Manuscript review: PM, SK, RG, MA, SupM, PKC, GRM, SunM, SKR, AR, KY, MB, PH. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Puneet Misra.

Ethics declarations

Ethics approval and consent to participate

The study received ethical clearance from institute ethics committee (Letter No. For AIIMS, New Delhi: IEC-959/04.09.2020). Written informed consent, assent and consent from the parents/guardians for the participants under the legal age of giving consent was obtained from all the participants as per ICMR guidelines. All the procedure was conducted as per the declaration of Helsinki and relevant guidelines.

Consent for publication

Not applicable.

Competing interests

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Misra, P., Kant, S., Guleria, R. et al. Test concordance and diagnostic accuracy of three serological assays for detection of anti-SARS-CoV-2 antibody: result from a population-based sero-epidemiological study in Delhi. BMC Infect Dis 22, 915 (2022). https://doi.org/10.1186/s12879-022-07805-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12879-022-07805-5

Keywords