The transition to Xpert MTB/RIF ultra: diagnostic accuracy for pulmonary tuberculosis in Kampala, Uganda

Background The World Health Organization (WHO) has endorsed the next-generation Xpert MTB/RIF Ultra (Ultra) cartridge, and Uganda is currently transitioning from the older generation Xpert MTB/RIF (Xpert) cartridge to Ultra as the initial diagnostic test for pulmonary tuberculosis (TB). We assessed the diagnostic accuracy of Ultra for pulmonary TB among adults in Kampala, Uganda. Methods We sampled adults referred for Xpert testing at two hospitals and a health center over a 12-month period. We enrolled adults with positive Xpert and a random 1:1 sample with negative Xpert results. Expectorated sputum was collected for Ultra, and for solid and liquid culture testing for Xpert-negative patients. We measured sensitivity and specificity of Ultra overall and by HIV status, prior history of TB, and hospitalization, in reference to Xpert and culture results. We also assessed how classification of results in the new “trace” category affects Ultra accuracy. Results Among 698 participants included, 211 (30%) were HIV-positive and 336 (48%) had TB. The sensitivity of Ultra was 90.5% (95% CI 86.8–93.4) and specificity was 98.1% (95% CI 96.1–99.2). There were no significant differences in sensitivity and specificity by HIV status, prior history of TB or hospitalization. Xpert and Ultra results were concordant in 670 (96%) participants, with Ultra having a small reduction in specificity (difference 1.9, 95% CI 0.2 to 3.6, p=0.01). When “trace” results were considered positive for all patients, sensitivity increased by 2.1% (95% CI 0.3 to 3.9, p=0.01) without a significant reduction in specificity (− 0.8, 95% CI − 0.3 to 2.0, p=0.08). Conclusions After 1 year of implementation, Ultra had similar performance to Xpert. Considering “trace” results to be positive in all patients increased case detection without significant loss of specificity. Longitudinal studies are needed to compare the benefit of greater diagnoses to the cost of overtreatment.


Background
Nucleic acid amplification testing (NAAT) is recommended by the World Health Organization (WHO) as an initial test for pulmonary TB. In particular, WHO has endorsed Xpert MTB/RIF (Cepheid, Sunnyvale, CA USA), and 34.4 million Xpert MTB/RIF (Xpert) cartridges have been procured globally [1]. However, Xpert sensitivity has been sub-optimal for smear-negative pulmonary TB [2]. Consequently, the next-generation cartridge, Xpert MTB/ RIF Ultra (Ultra), includes two multi-copy amplification targets, a larger DNA reaction chamber, and fully nested nucleic acid amplification to lower the limit of detection from 113 colony forming units (cfu)/mL to 16 cfu/ml, which is similar to growth detection [2]. Based on a multicenter assessment demonstrating increased sensitivity [3], the WHO recommended that Ultra replace Xpert as the initial diagnostic test [4].
Ultra cartridges are now expected to be widely used, including with the Gene Xpert Edge and Omni systems in lower-level facilities where patients more often seek initial medical care [5]. While the hope is that the lower limit of detection will increase case detection of patients with less advanced disease, there is also concern for reduced specificity and increased false-positive results, particularly among the category of "trace" results or in patients with past tuberculosis history [6].
During Uganda's transition from Xpert to Ultra, we conducted a 12-month cross-sectional study to 1) assess the diagnostic accuracy of Ultra for pulmonary TB in adults, 2) determine changes in accuracy by HIV, prior history of TB, and inpatient status, and 3) characterize the impact of trace results on Ultra accuracy.

Study design and population
From April 2018 to April 2019, we conducted a cross sectional study of patients with presumed pulmonary TB at Kiruddu Hospital pulmonary ward and outpatient chest clinic, Mulago Hospital TB ward, and Kisenyi Health Center IV outpatient clinic in Kampala, Uganda. As previously described [7], we included adults (> 18 years) who were referred by clinicians for sputum-based TB testing (i.e., Xpert). We excluded participants who received anti-TB treatment or antibiotics with anti-TB activity such as fluoroquinolones in the prior 12 months, or who refused or were unable to provide informed consent. To select participants, we reviewed the facility Xpert testing log daily. On each day, we enrolled all participants with Xpert-positive results and a random 1:1 sample of Xpert-negative patients identified from the log. The study and full protocol (available on request) was reviewed and approved by the Makerere University School of Medicine Research and Ethics Committee, the Uganda National Council for Science and Technology and the University of California San Francisco Committee on Human Research. This study was performed according to the Standards for Reporting of Diagnostic Accuracy Studies (STARD) guidelines [8].

Study procedures
After consent, all eligible participants completed a survey on demographics and medical history. Each participant provided up to two sputum samples: one for Ultra testing (all participants) and one for solid (Lowenstein-Jensen [LJ]) and liquid (BACTEC MGIT 960, Becton Dickinson, USA) mycobacterial cultures (Xpert-negative participants only). Positive cultures were confirmed using Ziehl-Neelsen (ZN) staining and M. tuberculosis (MTB) complex speciation testing (MPT64TB Ag kit, SD Bioline, South Korea). Trained laboratory technologists conducted all TB testing following standard protocols for Xpert, Ultra [2], and mycobacterial culture [9]. Laboratory staff were blinded to Xpert results at the time of Ultra or culture testing. Participants with negative or unknown HIV status underwent HIV testing per national guidelines, and CD4 cell count was measured for HIV-positive participants. To assess any additional microbiological evidence for false-positive results, Determine TB LAM (Alere, Waltham, USA) was performed on urine samples following standard protocols [10].

Definitions
We used a composite microbiological reference standard. Participants were defined as having pulmonary TB if MTB was detected by sputum Xpert or mycobacterial culture (solid or liquid). We considered patients not to have pulmonary TB if Xpert was negative and at least two culture results (solid or liquid) were negative without contamination. For the primary analysis, an Ultratrace result was defined as TB if the participant was HIV positive per WHO recommendations [11].

Statistical approach
Descriptive statistics were used to summarize the demographic and clinical characteristics of study participants. The sensitivity and specificity of sputum Xpert Ultra was calculated with exact binomial 95% confidence interval (CI) using our reference standard definition of TB. We compared sensitivity and specificity by HIV status, CD4 cell count category, prior history of TB, and inpatient status using Chi-squared or Fisher's Exact test as appropriate. We conducted a sensitivity analysis on Ultra trace results, comparing diagnostic accuracy if trace results were considered positive for all participants or no participants using McNemar's paired test of proportions. We used STATA 15 (Stata Corp, College Station, TX, USA) to perform all analyses.
Among patients with positive Xpert and Ultra results, rifampicin resistance results were concordant in 285/297 (96%) patients. Three patients had rifampicin resistance detected by Ultra but not Xpert, and one patient had rifampicin resistance detected by Xpert but not Ultra. Rifampicin resistance results were indeterminate in two patients by Ultra (all due to trace results, Xpert RIF negative), and in one patient by Xpert (Ultra RIF negative). Two of the discordant results were missing on Ultra and were not resistant on Xpert, while the three results missing on Xpert were not resistant on Ultra.

Subgroup analysis of ultra performance
As shown in Table 2

False-positive and trace ultra results
Of the seven patients with false-positive Ultra results, four were HIV-positive (including one with CD4 cell   (Table 3) We examined how classification of trace results impacted Ultra accuracy (Table 4). When trace results were defined as positive for all patients, Ultra sensitivity and specificity were 92.6% (95% CI 89.2 to 95.1) and 97.2% (95% CI 95.0 to 98.7), respectively, representing a 2.1% increase in sensitivity (95% CI 0.3 to 3.9, p=0.01) and a non-significant 0.8% reduction in specificity (95% CI − 0.3 to 2.0, p=0.08) compared to when trace results were considered to be positive only in HIV-positive patients as per WHO guidelines. When trace results were defined to be negative for all patients, Ultra sensitivity and specificity were 88.4% (95% CI 84.5 to 91.6) and 99.2% (95%CI 97.6 to 99.8), respectively, representing a 2.1% reduction in sensitivity (95% CI − 3.9 to − 0.3, p=0.01) and non-significant 1.1% increase in specificity (95% CI − 0.2 to 2.5, p=0.05) compared to the current guidelines.

Discussion
As Uganda is transitioning from Xpert to Ultra, we found that Xpert Ultra had high sensitivity and specificity for pulmonary TB overall (90.5% sensitive and 98.1%  Solid or liquid culture positive specific) and across key sub-groups defined by HIV status, prior history of TB, and hospitalization. Expanding the definition of a positive test to include trace results among HIV-negative patients increased Ultra sensitivity without a significant loss of specificity. Our findings suggest that as Ultra replaces Xpert in Kampala for routine TB testing at health facilities, diagnostic accuracy may remain stable overall, but that expanding the definition of a positive test to include all trace results could increase case detection. The accuracy of Ultra for pulmonary TB diagnosis was similar to previous studies. A recent meta-analysis of 10 studies found that Ultra could detect pulmonary TB with a sensitivity of 88.5% and specificity of 96.7% [12]. Among HIV-positive patients, we also found a similar Ultra sensitivity to other studies in HIV-endemic settings that found an 87-90% sensitivity without significant difference compared to HIV-negative patients [3,13]. Some studies have found that a previous history of TB reduced Ultra specificity [3,14,15], whereas we did not perhaps because we excluded patients who had TB in the past 1 year. The role of hospitalization has not been previously evaluated and we did not find a significant difference in inpatients vs. outpatients.
We did not find a significant difference in sensitivity of Xpert vs. Ultra. This is in contrast to a multicenter study that found Ultra was 5.4% more sensitive overall and 13% more sensitive in HIV-positive patients [3]. The lower yield in our study is likely due to the small proportion of culture-positive patients that were not detected by Xpert (for example, 7% in our study compared to 17% in Dorman et al. [3]), and potentially the inclusion of Xpert in our reference standard. At the same time, multiple studies in Brazil and South Africa also had high concordance between Ultra and Xpert without a significant difference in overall sensitivity [13,15,16]. Nonetheless, this finding is surprising given that we had a high prevalence of HIV-positive patients and outpatients, both of whom would be expected to have lower bacillary burden and greater yield with Ultra over Xpert. In addition, 5% of culture-confirmed cases were both Ultraand Xpert-negative. This underscores the continued need for novel diagnostics that can increase the yield of TB case detection in patients with paucibacillary disease.
The reduction in specificity of Ultra vs. Xpert has been documented in several studies [3,13,15,17]. Four of seven false positive results were trace results among HIV positive patients, of which one had additional evidence of TB disease with a positive Determine TB LAM result. While this is consistent with past studies that found the majority of false positives were due to trace results [3,13], we had few overall false positives and only three had a prior history of TB. As a consequence, we did not find a significant increase in specificity when trace results were defined as negative. Conversely, expanding the definition of a positive test to include trace results in HIV-negative patients increased sensitivity without a significant loss of specificity, because the majority of HIV-negative patients with trace results had microbiologically-confirmed TB.
The decision to define trace results as positive or negative requires a balance between greater case detection and cost of overtreatment [6]. A modeling study found that while Ultra trace results increased overtreatment by more than 50%, it also averted deaths by that amount [18]. The authors further concluded that Ultra and trace results will have the greatest benefit in settings with high HIV and TB prevalence and TB mortality. As Uganda is one of those settings, further costeffectiveness studies are needed to guide the role of trace results, especially in facilities where health workers may not have access to culture or other testing to confirm the diagnosis.
To evaluate the performance of Ultra in Uganda, we enrolled a sample from three major medical centers in Kampala with high HIV prevalence [19]. However, this study has some limitations. We did not confirm positive sputum Xpert results with culture, and did not confirm rifampin resistance with drug susceptibility testing. However, Xpert has been shown to have very high specificity (> 98%) for TB and rifampin resistance [20]. As smear microscopy was not done on Xpert positive samples, we could not stratify Xpert Ultra performance by smear status. We also did not have the ability to have follow-up visits to repeat Ultra testing on trace results to assess additional yield. In addition, follow-up visits would have been helpful to further characterize patients with both false-positive and trace Ultra results.