Study-based evaluation of the Abbott RealTime High Risk HPV test in comparison to the HC2 HR HPV test in women aged ≥30 years using residual LBC ThinPrep specimens

Background High-risk human papillomavirus (HR HPV) testing is already part of cervical cancer screening programs in a number of countries. New tests need to be validated not only in clinical studies but also in routine screening settings with regard to their clinical performance. Methods The Abbott RealTime High Risk HPV Test (RT hrHPV test) was evaluated in a random sample of 1,456 patients from a German routine screening population of 13,372 women ≥30 years of age screened primarily by liquid-based cytology (LBC) that was complemented by 48 CIN3+ cases. Clinical sensitivities, relative specificities and positive predictive values (PPV) for both HPV tests were determined based on histologically confirmed high-grade cervical disease (CIN3+) as clinical outcome. Results HR HPV prevalence in residual LBC samples was found to be 5.4 % by the RT hrHPV test and 5.6 % by the HR HC2 test, respectively. The Kappa-value for overall agreement between the RT hrHPV test and the HC2 assay for detection of HR HPV was 0.87. Relative sensitivities for detection of CIN3+ in patients with abnormal cytology was 93.8 % for the RT hrHPV assay and 97.9 % for HC2 (p-value = 0.5). Relative specificities and PPVs were comparable for both tests. The highest PPV was calculated for the specific detection of HPV16 by the RT hrHPV test (84.2 %). The RT hrHPV test showed a reduced sensitivity for detection of HVP31-positive CIN3 + . Conclusion The RT hrHPV assay is as sensitive and specific in detecting severe cervical lesions in women with abnormal cytology as the HC2 HR HPV test.


Background
Since the introduction of opportunistic cytological screening in Germany in 1971 the cervical cancer mortality rate has notably decreased [1]. However, 4,600 new cases and approximately 1,500 deaths due to cervical cancer are diagnosed in Germany each year [2]. Moreover, 150,000 cases of cervical cancer precursors (CIN3) are detected annually [3] and cervical cancer is the cause for 1.5 % of all female cancer deaths in Germany [2]. As persistent infection with High-Risk Human Papillomaviruses (HR HPV) is a necessary risk factor for the development of (pre)-cancer, numerous HPV tests are nowadays commercially available [4] to be used in cervical cancer screening programs [5]. However, only a small number of these tests have been approved by the FDA [6] with the Digene Hybrid Capture 2 High-Risk HPV DNA test as the first one (HC2; QIAGEN Hilden, Germany). The HC2 has been developed for the collective detection of 13 carcinogenic HPV types (16,18,31,33,35,39,45,51,52,56,58,59,68) [7]. It is one of the best validated HPV tests whose methodology is based on nucleic acid hybridization with signal amplification for qualitative detection of HPV-DNA within cervical samples.
The Abbott RealTime High Risk HPV Test (RT hrHPV) is another fully automated HPV DNA test which is based on real-time PCR that targets the L1 region of the 13 carcinogenic HPV types 16,18,31,33,35,39,45, 51, 52, 56, 58, 59, 68 and for detection of HPV 66. The multiplex design of the assay allows HPV16 and 18 genotyping as well as the collective detection of the 12 remaining HPV types [8]. The RT hrHPV test has been fully validated by several cross-sectional studies that evaluated its clinical performance in referral populations [9][10][11][12][13][14][15][16][17][18]. However, to date only four screening population-based cross sectional studies [19][20][21][22] and one follow-up study are available [23]. The objective of this retrospective study was the cross-sectional evaluation of the RT hrHPV assay in a routine cervical cancer screening population comprising women aged ≥ 30 years in Germany.

Methods
The study cohort and methodology has previously been published [24].

Study design
Briefly the study was conducted on a sample of 2,303 women of a routine screening population of 13,372 women ≥ 30 years of age living in the Hannover area of Germany in 2011 (Fig. 1). Cervical samples were collected in PreservCyt® Pap Test specimen collection medium (Hologic) and cytology was tested within 1 week after collection in a central services laboratory (Amedes, Bad Münder, Germany). After exclusion of ineligible samples, a total of 1,456 residual liquid based cytology (LBC) smears from this cohort including all samples with abnormal and ASC-US cytology results as well as 10 % of randomly selected samples with normal cytology results were tested by both HR HC2 and Abbott RealTime High Risk HPV Test (RT hrHPV). This collection was complemented with samples from 48 patients with CIN3+ from a separate referral cohort in order to obtain a sufficient CIN3+ rate. HC2 tests were RT hrHPV assays were performed in the Amedes laboratory within 2 weeks after collection. Samples with discordant HPV test results were genotyped by INNO-LiPA HPV Genotyping Extra® for discrepancy analysis.
Samples were anonymized thus, participants were unaware of their HPV test results and colposcopy as well as histopathology was only performed when indicated by German standard operating procedures. Samples with a primary histology of ≥ CIN2+ were independently reviewed by up to two pathologists.

Sample collection
Sample Collection has previously been described [24]. Briefly cervical samples were collected in LBC Preserv-Cyt® Collection medium (Hologic) using the Cervex broom according to routine guidelines. Samples were then centrally tested by cytology within one week of collection.
Liquid based cytology LBC was performed as described before [24] by the ThinPrep® 2000 Processor (Hologic) following the manufacturer's instructions. Cytology results were reported using the Munich Nomenclature II and were translated into The Bethesda System (TBS) ( [25]; Table 1). LBC results were considered negative when the result was Pap I/II (Normal) or Pap IIw (ASC-US); all other results were considered positive [24].
HPV testing and genotyping LBC samples were tested by Abbott RealTime High Risk HPV Test (RT hrHPV) in compliance with the manufacturer's instructions. All specimens included in this study were initially tested by RT hrHPV test and subsequently analyzed by Digene Hybrid Capture 2 High-Risk HPV DNA (HC2) test. HC2 testing has previously been described [24].

Histology reviews
As detailed before all samples with an initial histology result of ≥ CIN2+ were reviewed by an independent external expert. In the case of a discrepant review reading, a second histology review was performed. If two out of three diagnoses were identical, the result was considered final [24].

Statistical analysis
As described [24] statistical analysis was performed on all samples with valid LBC, RT hrHPV and HC2 (N = 1,456) test results. To calculate the agreement between RT hrHPV and HC2, the Cohen's kappa value (κ) was used. The Wilson score method was used to calculate 95 % confidence intervals (CI) for HPV prevalence. Moreover, clinical sensitivity and relative specificity as well as positive predictive values (PPV) and negative predictive values (NPV) were calculated according to Cuzick et al. [20] based CIN3+ histology results.
Relative performance of the two tests was measured by calculating the ratio of the sensitivity to the specificity. This ratio of the sensitivity of the two tests was defined as the True Positive rate of the first test divided by the True Positive rate of the second test. The relative specificity on the other hand, is dependent of prevalence, and was expressed as Spec(RT hrHPV)/Spec(HC2) = (0.818-Prevalence)/(0.800 -Prevalence). The delta method was used to determine confidence intervals. A full description of this method has previously been published [24]. Statistical analysis of the relative sensitivity and specificity was calculated using the statistics software package R version 3.0.2.

Results
Of 2,303 overall specimens a total of 847 samples had to be excluded from the analysis either because they returned unsatisfactory cytology results (n = 196), because they could not be analyzed by all three tests due to insufficient material (n = 547) or because study participants were younger than 30 years of age (n = 104 A total of 65 discordant samples were detected between the RT hrHPV and HC2 (Table 3) (Table 3). As a result we resolved 5 RT hrHPV-negative and 7 HC2-negative samples as true negatives. While four samples were inadequate for LiPA genotyping, all remaining samples (n = 49) were HPV DNA positive.
In detail one sample missed by HC2 contained a nontarget type of the HC2 assay and a total of 9 specimens (50 %) were false-negative by HC2. However, none of the samples missed by HC2 had a histology result of CIN3+. Interestingly, the HC2 assay detected HPV DNA in 18 RT hrHPV-negative samples, which contained non-target types of either HPV DNA test including 11 cases of HPV 53.
In contrast 18 discordant samples (38.3 %) with negative RT hrHPV and positive HC2 test results were nontarget types of the RT hrHPV test (including 2 HPVX  Fig. 2. The overall detection rate for HPV 16 (6.8 %) was higher than that of HPV 18 (1.0 %). Prevalence of HPV 16 and 18 increases proportionally with higher grade of cytology abnormalities and was highest in HSIL and CIS (≥Pap IVa). 58.5 % of these samples were positive for HPV 16 and 12.2 % for HPV 18. In contrast HPV 16 prevalence was 2.2 % compared to 0.2 % for HPV18 in the normal (Pap I/II) and 3.6 % versus 0 %, respectively, in the ASC-US (Pap IIw) category. HPV 16 and HPV 18 were both detected in 6.3 % of the specimens with ASC-H, AGC (Pap III) cytology results in contrast to the LSIL, HSIL (Pap IIID) category, which included 19.9 % HPV 16 positive and 2.3 % HPV 18 positive samples. All samples with a histology result of CIN3+ were genotyped by using the INNO-LiPA HPV Genotyping Extra test. Overall HR HPV genotype distribution in specimens with CIN3+ lesions is shown in Fig. 3. All HPV types detected in both single and multiple infections were included in the calculations. Not surprisingly, HPV 16 (46.2 %) was the most prevalent genotype in specimens with CIN3+ lesions followed by HPV 31 (12.3 %), HPV 52 (9.2 %), HPV 33 (6.2 %) and HPV 18 (4.6 %). All specimens with a LiPA Extra genotyping result of either HPV 16 or HPV18 were correctly identified by the concurrent Abbott RT HPV 16/18 genotyping.

Sensitivity and specificity
Relative sensitivity and specificity were calculated to define the diagnostic accuracy of the HR HC2 and the RT hrHPV test for detection of CIN3+. Within this cohort 81 biopsies were available from patients with abnormal cytology results. Overall relative sensitivity for detection of CIN3+ in patients with abnormal cytology was 93.8 % Colposcopy was given only to women with abnormal cytology findings; therefore the true number of high grade CIN in the normal (Pap I/II) and ASC-US (Pap IIw) categories is not known. Due to this lack of definitive data one could not calculate the clinical sensitivity and specificity of the two tests. However, we calculated the ratio of the sensitivity and specificity of the two tests to each other, thus we are able to determine relative, if not absolute, performance. The ratio of sensitivities between RT hrHPV test and HC2 is 0.959 (95%CI: 0.885-1.033). Fig. 4 shows the ratio of specificity as a function of prevalence. We found no statistical difference between clinical sensitivity and specificity of the RT hrHPV test and HC2 for the detection of high grade disease at the 95 % confidence level.

Discussion
This study was conducted to evaluate the analytical and relative performance of the RT hrHPV test using residual LBC specimens selected from a German routine cervical cancer screening population. HR HPV positivity rates overall (23.4 % for HC2 and 21.4 % for RT hrHPV) and in LBC normal samples (5.4 % for RT hrHPV and 5.6 % for HC2) were in line with previously published data [21]. HPV prevalence in women with abnormal cytology (AGC+ ≥ PapIII) was 68 % detected by RT and 71.3 % for the HC2 test. The generally slightly higher detection rates of the HC2 test may be attributed to crossreactivity with non-target types [27]. The most prevalent HPV type in women with high grade cervical disease in this study was HPV 16, which is in accordance with all recent studies and meta-analyses (summarized by [28]).
Comparing the performances of both tests we showed that the RT hrHPV and HC2 test performed similarly and the agreement of both assays was excellent (κ = 0.87). These results reflect previously reported data [13,19,21] and indicate that both HPV DNA tests performed equivalently.  Discordant samples were analyzed by the LiPA HPV genotyping Extra test, which has been used as an adjudicating assay in test comparison studies before [24,29,30], due to its high analytical sensitivity [31]. We found that 50 % of HC2-negative, but RT hrHPV-positive samples were true HPV negative. The possibility of false-negative HC2 results has previously been reported to be attributed to a low viral copy number leading to false-negative HC2 results [32] or to the lack of an internal control for cellularity, which is however ruled out here by the split sample protocol. Furthermore, we found that the HC2 test was able to detect infections with the non-target type HPV 53. Cross-reactivity of the HC2 test with HPV 53 has been demonstrated by multiple previous reports. In fact, HPV 53 has been shown to be one of the most frequent non-target types detected by the HC2 test through cross-hybridization of its HR probe [21,[33][34][35]. While the HC2 test detected all cases of CIN3 +, the RT hrHPV test missed two CIN3+ cases positive for HPV 31. Similar results have previously been published by Poljak et al., who reported that the RT hrHPV test missed two CIN3 + −cases with HPV 31 and 58 co-infections [36]. Two other reports also demonstrated a diminished sensitivity of the RT hrHPV test for HPV 31-positive CIN3 cases [21,30]. Indeed in the present study we found that one of the two CIN3+ cases missed by RT hrHPV test represented a co-infection with HPV-type 33, which might indicate competitive primer binding in the PCR leading to an unfavorable kinetic of amplification for HPV 31 or other types from the alpha-9 subgenus in mixed infections.
Comparing concurrent genotyping results of the RT hrHPV test we were able to show that RT HPV 16/18 genotyping correctly identified all specimens with a LiPA Extra genotyping result of either HPV 16 or HPV 18. These results are in contrast with earlier reports showing concordance between LiPA and RT genotyping in only 90 % of the tested specimens [10].
Relative sensitivities for detection of CIN3+ were high and comparable for both tests (p-value = 0.5) whereas relative specificities for RT was slightly higher than for the HR HC2 test. By calculating the ratios of sensitivities and specificities, respectively, we were able to confirm that no statistical difference exists between the two tests' performances. Our observations are in line with cross-sectional studies reported previously in routine screening populations [19][20][21] and suggest that RT hrHPV test is well suited to be used in routine primary cervical cancer screening or adjunctive to cytology. Further evidence for the applicability of the RT hrHPV test in primary screening has recently been published by Poljak et al., reporting the first longitudinal data for the Abbott RT HPV test [23]. The authors demonstrated the non-inferior clinical performance of the RT HPV test in a routine screening population of 3,920 women with a 3-year follow-up time in comparison to the performance of the HC2 test.
In summary we found that the sensitivities and relative specificities of the RT hrHPV and the HR HC2 test are comparable. However, it appears that the RT hrHPV test has a reduced sensitivity for the detection of HPV type 31, which in this study has led to two CIN3+ cases missed by the RT hrHPV test. Because HPV 31 is one of the most prevalent high-risk HPV-types worldwide [37] and is found in 3,5 % of global cervical cancer cases [38], it is important to carefully assess risks and benefits of applying the RT hrHPV test. A considerable benefit of the RT hrHPV HPV test is its separate type specific detection of the most prevalent HPV genotypes 16 and 18, which together account for a total of an estimated 70.4 % of cervical cancer cases worldwide [38]; and we demonstrated that the PPV for detecting high grade disease is by far highest for Abbott RT hrHPV HPV 16.

Conclusions
We provided evidence that the Abbott HR HPV test is suitable for primary routine screening in Germany and other countries with a similar infrastructure regarding the secondary prevention of cervical cancer.

Funding
Not applicable.
Availability of data and materials Anonymous data are available from the corresponding author on reasonable request.
Authors' contributions TI designed the study, was involved in data analysis and interpretation and drafted the manuscript. LW and PM performed the Ratio of specificities and sensitivities analysis and were involved in drafting the manuscript. BH was the studies coordinator, performed statistical analyses and drafted the manuscript. AI and NI performed HPV testing and genotyping. JHJ wrote the manuscript. RvW performed histologies. GB recruited study participants, collected the samples and performed LBC. All authors read and approved the final manuscript.

Competing interests
This study was supported by an unconditional grant from Hologic Incorporated to the University Hospital of Tübingen, which hosts the Division of Experimental Virology. Hologic had no influence on the study protocol, no access to primary data and was not involved in the analysis and writing of the manuscript.

Consent for publication
Not applicable.
Ethics approval and consent to participate As determined by the ethics committee (Ethikkommission bei der Deutschen Ärztekammer Niedersachsen), no ethical approval or consent was required for this study, because the data was collected anonymously.