Evaluation of Xpert MTB/RIF and MODS assay for the diagnosis of pediatric tuberculosis

Background Tuberculosis (TB) in children is rarely confirmed due to the lack of effective diagnostic tools; only 10 to 15% of pediatric TB is smear positive due to paucibacillary samples and the difficulty of obtaining high-quality specimens from children. We evaluate here the accuracy of Xpert MTB/RIF in comparison with the Micoroscopic observation drug susceptibility (MODS) assay for diagnosis of TB in children using samples stored during a previously reported evaluation of the MODS assay. Methods Ninety-six eligible children presenting with suspected TB were recruited consecutively at Pham Ngoc Thach Hospital in Ho Chi Minh City Viet Nam between May to December 2008 and tested by Ziehl-Neelsen smear, MODS and Mycobacterial growth Indicator (MGIT, Becton Dickinson) culture. All samples sent by the treating clinician for testing were included in the analysis. An aliquot of processed sample deposit was stored at −20°C and tested in the present study by Xpert MTB/RIF test. 183 samples from 73 children were available for analysis by Xpert. Accuracy measures of MODS and Xpert were summarized. Results The sensitivity (%) in detecting children with a clinical diagnosis of TB for smear, MODS and Xpert were 37.9 [95% CI 25.5; 51.6], 51.7 [38.2; 65.0] and 50.0 [36.6; 63.4], respectively (per patient analysis). Xpert was significantly more sensitive than smear (P=0.046). Testing of additional samples did not increase case detection for MODS while testing of a second sputum sample by Xpert detected only two additional cases. The positive and negative predictive values (%) of Xpert were 100.0 [88.0; 100.0] and 34.1 [20.5; 49.9], respectively, while those of MODS were 96.8 [83.3; 99.9] and 33.3 [19.6; 49.5]. Conclusion MODS culture and Xpert MTB/RIF test have similar sensitivities for the detection of pediatric TB. Xpert MTB RIF is able to detect tuberculosis and rifampicin resistance within two hours. MODS allows isolation of cultures for further drug susceptibility testing but requires approximately one week to become positive. Testing of multiple samples by xpert detected only two additional cases and the benefits must be considered against costs in each setting. Further research is required to evaluate the optimal integration of Xpert into pediatric testing algorithms.


Background
The World Health Organisation has estimated that there are 1 million cases of tuberculosis in children every year [1]. Seventy-five percent of these cases occur in the 22 high TB burden countries [2]. TB in children is rarely confirmed due to the lack of effective diagnostic tools; only 10 to 15% of pediatric TB is smear positive due to paucibacillary samples and the difficulty of obtaining high-quality specimens from children [3]. HIV coinfection further complicates diagnosis [4]. The overall focus of National TB programmes on notification of smear positive tuberculosis under the DOTS strategy has thus led to a 'chronic neglect' of pediatric TB, under the assumption that an overall decline in adult tuberculosis would also address pediatric TB [5,6]. Current best estimates of the burden of childhood TB are based upon multiple assumptions and the true burden of pediatric TB is unknown.
Diagnosis of pediatric TB is principally based on the evaluation of medical history, clinical evaluation with chest radiography, tuberculin skin-testing and smear microscopy with mycobacterial culture where available [7,8]. Diagnostic tests currently used for adult tuberculosis show low sensitivity when applied to children. Until the advent of the Xpert MTB/RIF test, nucleic acid amplification tests (NAAT) required a specialized laboratory with rigorous quality control which can be difficult to maintain in low resource settings. In the last decade, significant advances have been made in the development and implementation of novel diagnostic tests for TB in adults [9]. However, evaluation of the applicability of these tests for pediatric TB has not been extensive [6].
The non-commercial rapid liquid culture technique, microscopic observation drug susceptibility (MODS) assay [10] has a sensitivity for detection of TB in children similar to Mycobacterial growth indicator (MGIT) culture with faster turn-around times and can therefore significantly increase confirmation of TB in children compared to smear microscopy alone [11,12]. However, the plate reading is relatively labour intensive and cultures require approximately a week to become positive.
The Xpert MTB/RIF test (Cepheid, USA) has shown sensitivity and specificity approaching that of culture in adult sputum samples, 90.4% sensitivity and 98.4% specificity in a recent meta-analysis and has been endorsed by the WHO for use on sputum samples [13,14].
Scale-up implementation projects are underway in a number of countries, including Vietnam [15]. A recently published study of 452 children with suspected TB in South Africa, a high HIV incidence region, reported a doubling of the case detection with Xpert MTB/RIF compared to fluorescence smear, detecting 78% and 38%, respectively [16]. The test is a promising method to improve the diagnosis of pediatric TB, but limited data is available on the performance of the test with pediatric samples and is urgently required to determine appropriate applications for Xpert testing.
Vietnam is a high TB burden country with an estimated incidence of 200/100 000 population and an estimated 2.7% prevalence of MDR TB among new cases [17]. Twenty five percent of the Vietnamese population is under 15 years old [18]. In 2007, only 14.9% (59/395) of treated pediatric TB cases had microbiological confirmation at Pham Ngoc Thach hospital, the tertiary referral hospital for TB in southern Vietnam [11].
We evaluate here the accuracy of Xpert in comparison with the MODS assay for diagnosis of TB in children using samples stored during a previously reported evaluation of the MODS assay [11].

Recruitment
This study used stored samples and data from a previously reported prospective evaluation of the MODS assay for diagnosis of pediatric TB [11]. The protocol was approved by the Institutional Review Board (IRB) at Pham Ngoc Thach Hospital and the Health Services of Ho Chi Minh City. Informed consent was not sought because the study was conducted on routine samples only and it did not involve any intervention, additional samples or change in patient management. This patient consent waiver was approved by the IRB of Pham Ngoc Thach Hospital in the protocol.
All eligible children <16 years of age with clinical suspicion of tuberculosis presenting to the pediatric ward at Pham Ngoc Thach Hospital, Ho Chi Minh City, Viet Nam from May 2008 to December 2008 were included in the study [11]. Any patient already receiving TB therapy for more than seven days was excluded from the study.
Children were classified into 3 diagnostic categories, confirmed TB, probable TB or TB unlikely [11]. Children with confirmed TB had acid-fast bacilli observed by smear microscopy or M.tuberculosis isolated from a clinical sample by MGIT culture. Children with 'probable TB' had clinical symptoms consistent with TB, did not receive any alternative diagnosis and received TB treatment. Children in the 'TB unlikely' category recovered without TB treatment or received an alternative confirmed diagnosis.
The treating clinician determined the number and type of samples collected, following routine practice. No additional samples to routine care were collected as part of this study.

Sample processing
During the original evaluation of the MODS assay, an aliquot of decontaminated sample deposit was stored at −20°C. These stored aliquots were used for current evaluation of Xpert MTB/RIF. In total, 183 samples from 73 patients were available for analysis, including sputum (n=126), gastric fluid (n=49), cerebral spinal fluid (CSF) (n=5), and pleural fluid (n=3). All of these samples, except CSF, were processed by standard decontamination using Sputaprep (NaOH -NALC 2%, Nam Khoa Company-Viet Nam) before storage.
Smear, MGIT and MODS were performed as described previously in accordance with standard protocols.
Ziehl-Neelsen (ZN) smear: Two drops (approximately 200 μl) of processed sample deposit were placed on a slide and stained by ZN method according to the WHO standard protocol [19].
MGIT: Five hundred microlitres of each deposit was inoculated into a supplemented MGIT tube following the manufacturer's protocol. The culture was incubated in Bactec MGIT 960 system at 37°C and the result automatically reported.
MODS assay: MODS culture was prepared in a 48well-tissue-culture plate. Two hundred and fifty microlitres of pellet was added to 7H9 medium supplemented with OADC and PANTA antibiotic solution. After at least 4 days of incubation, the growth of M. tuberculosis was recorded by reading through an inverse microscope.
Xpert MTB/RIF: The test was conducted on decontaminated sample deposits following the manufacturer's standard operating procedure. The sample volume stored varied from 0.5 ml-1.5mls. Processed sample deposits exceeding 0.5 ml were centrifuged at 3,000 g for 15 minutes and excess supernatant discarded to obtain a final volume of 0.5 ml. One and a half milliliters of sample reagent was added to 0.5 ml of processed sample, and incubated at room temperature for 15 minutes with intermittent shaking and finally added to the test cartridge and loaded onto the machine. The result was available after 2 hours.
For any sample with a positive result for rifampicin resistance by Xpert MTB/RIF, the corresponding isolate from the same sample isolated by MGIT or MODS culture was tested for RIF and isoniazid (INH) susceptibility by MTBDRplus test (Hain Lifesciences, Germany) and Bactec Mycobacterial growth indicator tube SIRE Drug susceptibility test (MGIT SIRE DST, Becton Dickinson, USA) following the manufacturer's protocol [20].

Statistical analysis
Accuracy measures (sensitivity, specificity, positive and negative predictive values) of the evaluated tests for M. tuberculosis detection were determined using two gold standards for the reference diagnosis. The first comparison was made using microbiologically confirmed TB (by smear or MGIT) as the gold standard. The second comparison was made using a clinical diagnosis of TB (i.e. confirmed or probable TB) as the gold standard. Accuracy measures and corresponding confidence intervals were reported both on the patient and on the sample level.
For the per patient analyses, a patient was considered to be positive if the test showed a positive result for at least one of the patient's samples. Exact Pearson-Clopper confidence intervals for accuracy measures are reported. Comparisons between the sensitivities of different diagnostic tests are based on McNemar's test with continuity correction.
For the per sample analyses, confidence intervals for accuracy measures and comparisons of sensitivities of different tests are based on marginal logistic regression with an identity link function. Of note, using marginal models and generalized estimating equations accounts for potential dependence between test results of samples from the same patient. To account for the fact that specificities and positive predictive values were close to 100% leading to a separation problem for the marginal logistic regression, we used an Agresti-Caffo-type adjustment for the confidence intervals in these cases [21]. We also calculated time-dependent sensitivity curves for MGIT and MODS. For these, a test result was considered as positive by time t if the respective test was positive overall and reached the positive value at most t days after sample collection. Time-dependent sensitivity curves were estimated with the Kaplan-Meier method and samples without a positive test result were formally regarded as censored on day "infinity".
The association between Ct values for probe A and smear grade was summarized and tested based on Spearman's rank correlation.
All analyses were done using the statistical software R version 2.14.0 (R Foundation for statistical computing, Vienna, Austria). All reported confidence intervals are two-sided 95% confidence intervals and p-values ≤ 0.05 were considered as statistically significant.

Results
The previously reported prospective evaluation of the MODS assay for diagnosis of pediatric TB [11] included data from 96 children but 23 children did not have a stored aliquot available for analysis by Xpert in the present study due to insufficient volume for storage or the use of the stored deposit aliquot for reculture in the original MODS study. The majority (69.6%, n=16/23) of these were in the probable TB category. Thus, data from 73 children were included in the present analysis ( Figure 1). Almost half of these children (n=35/73, 47.9%) were between 11 and 16 years old, with 20.5% (n=15/73) 6-10 years old and 31.5% (n=23/73) 0-5 years of age.
There were 31/73 children (42.5%) with confirmed TB, 27/73 (37.0%) with probable TB and 15/73 (20.5%) were TB unlikely. The treating clinician determined the number and type of samples collected. For children aged five years or lower, the most common sample type was gastric aspirate, while for older children, sputum was normally collected. Only four children had two different sample types (Table 1).
HIV testing was not performed as part of the study, and only eight children had an HIV test as part of routine clinical care, in accordance with normal practice at PNT. Seven of eight children tested were HIV infected.
Amongst 183 samples from 73 children tested, there were 2 (1.1%) Xpert tests with invalid reports and 3 (1.6%) with error reports ('post-run analysis' for 2 tests, 'operation terminated' for 1 test). Error and invalid results were excluded from further analysis, resulting in a total of 178 samples but this did not decrease the number of patients (n=73).

Accuracy of Xpert MTB/RIF Per patient analysis (n=73)
The majority of patients (86%, 63/73) submitted multiple samples; 60% (44/73) of patients produced 3 samples and 26% (19/73) produced 2 samples.   In the 'per patient' analysis, Xpert showed a very similar sensitivity to MODS for both confirmed TB alone and all TB cases (confirmed + probable), 93.5% vs. 96.8% and 50.0% vs. 51.7%, respectively. Xpert was significantly more sensitive than smear (p= 0.046, same p-value for both gold standards) ( Table 2). Relative to smear, seven additional cases were detected overall by Xpert while MODS detected eight additional cases over smear. Therefore, only a single case was detected by MODS which was not detected by Xpert.
The specificity of Xpert in all analyses was 100%. For MODS culture, the specificity was 93.3% for the per patient analysis of all TB cases due to a cross-contamination event identified by spoligotyping. Details of the contamination analysis for MODS have been previously reported [11].

Per sample analysis (n=178)
The sensitivity (%) of Xpert for detection of confirmed TB was 79. 5   For all TB cases (confirmed + probable), the sensitivity of 47.1% for MODS was also marginally higher than the sensitivity of 44.3% for Xpert, but the difference was not statistically significant (P=0.74). For all TB cases, the sensitivity of MGIT was 51.4% which was not significantly higher than Xpert (P=0.43). The difference in sensitivities between Xpert and smear (sensitivity 30.0%) did not reach statistical significance for all TB cases (P=0.09) ( Table 2).
Xpert detected 23 positive samples which were not detected by smear, while MODS detected 27. This difference in detection of smear-negative TB was not statistically significant (p=0.24).
Xpert was negative for M.tuberculosis in 7% (3/42) of smear-positive samples, all of which were 'scanty' by smear. Among them, one pleural fluid sample was also reported as negative by both culture methods. MODS also did not detect M.tuberculosis in 7% (3/42) of smear positive samples all of which were 'scanty'. Two of them were positive by Xpert (category very low and high).

Incremental sensitivity of MODS and Xpert using multiple sputum samples from individual patients
We investigated the cumulative sensitivity of Xpert on additional samples collected from all 24 TB cases (con-firmed+probable) who provided 3 sputum samples. Xpert detected M.tuberculosis in 50% (12/24) of first sputum samples. Testing a second sample by Xpert detected an additional two cases [+8.3%]. The sensitivity did not increase with a third sputum sample. MODS detected 62.5% (15/24) patients with the first sputum sample but did not detect any additional cases with a second or third sample.

Time to detection
Xpert requires 2.5 hours to return a positive result. To reach a comparable sensitivity to the 79.5% of Xpert for microbiologically confirmed TB, MGIT and MODS both required around 20 days. Although MGIT culture was the most sensitive test, it took 32 days for the last culture to turn positive and reach the per-sample sensitivity of 92.3%. In positive samples, the median time to detection of MGIT and MODS was 8 days (IQR 7-13) and 13 days (IQR 9-18), respectively.

Correlation of Ct value and smear grade
To investigate the correlation between quantification of bacterial load by conventional smear grading and Xpert MTB/RIF categorisation, we stratified the Ct value of probe A according to smear grade ( Figure 2).
We found a strong negative association between Ct values and smear grade with higher Ct values in samples with low bacterial load (Spearman rank correlation −0.84, p<0.001). The majority of positive samples had an Sensitivity by sample type for confirmed TB cases: Xpert showed the same sensitivity in both sample types in TB cases with microbiological confirmation, 80.3% (n=55/66) in sputum and 81.8% (n=9/11) in gastric fluid (per sample analysis).

Detection of Rifampicin resistance
Two patients were detected as infected with RIF resistant strains by Xpert. The test was consistently positive for RIF resistance for consecutive samples collected from the same patient.
MTBDR-Plus testing confirmed both strains as resistant to RIF (mutations at rpoB 526 and 531). Both strains were also resistant to both RIF and INH, and therefore MDR TB by MGIT SIRE. One of the two MDR strains was classified as INH susceptible by MTBDR-Plus. Drug resistance was not determined by MODS assay.
MTBDR-Plus was also performed on DNA extracted from 30 isolates from samples which were positive with M. tuberculosis/RIF susceptible by Xpert (one sample per patient). All were susceptible to RIF by MTBDR-Plus. There were therefore no discrepencies detected between Xpert RIF resistance detection and other tests. There was no sample with an 'indeterminate' result for RIF susceptibility by Xpert.

Discussion
Xpert MTB/RIF detected 50% of children with clinically diagnosed pediatric TB, a 12%-increase over smear. Testing of multiple samples did not increase case detection for MODS and testing of a second sputum sample by Xpert detected only an additional two cases. The liquid culture techniques MODS and MGIT did not have a significantly higher sensitivity and Xpert is a 'same-day' test while liquid culture requires at least one week to return positive results. For maximal detection of pediatric TB, the optimal approach appears to be pairing Xpert with liquid culture (MGIT where available or MODS as an alternative). Although culture only detected a single additional case in this study, the 'per sample' sensitivity of MGIT culture was 14% higher overall, and culture is required to enable full drug susceptibility testing. In addition, Xpert MTB/RIF is able to rapidly detect RIF resistance. However, in this study only two patients were infected with RIF resistant strains and no robust conclusions can be drawn about the accuracy of Xpert for detection of MDR in these paucibacillary samples. The implementation of new diagnostic tests is urgently required to improve case confirmation for pediatric TB and to reduce unnecessary treatment.
Sensitivity of Xpert for detection of pediatric TB was higher in this study than two previous studies from South Africa (20.3%) and Tanzania (33.3%) [16,22]. The higher proportion of smear positive TB in our study suggests there are important differences in the patient populations between these studies, leading to the different reported Xpert sensitivities. This is likely to be due to two principal factors. Firstly, the median age of children is higher in the present study (median 8.8 years this study, 6 years Tanzania, 19 months South Africa) and confirmation of pediatric TB is generally higher in older children. For example in this study, per sample sensitivity of smear, MODS and xpert was 7.7%, 23.1% and 23.1%, respectively for children aged less than four, and 48.8%, 62.8% and 60.5% respectively, for those aged four and above. Secondly, the present study was conducted in a tertiary referral hospital for TB rather than a general hospital, therefore a higher proportion of the children were microbiologically confirmed by all methods, probably due to more advanced disease and higher bacillary loads in the samples. All studies show that Xpert increases sensitivity detection of pediatric TB over smear techniques.
Our analysis of children with 3 sputum samples suggests that the incremental value of testing multiple sputum samples from children by Xpert is moderate, with an observed 8% increase (from 50% to 58%) in sensitivity from a second sample and none from a third. Given the relatively high costs of Xpert testing, it is likely that repeat testing is only beneficial when the index of suspicion is high in this population. Further research will be needed to determine the most cost-effective algorithmic approaches to Xpert testing in children. The optimal approach will vary by setting, depending upon available resources and the patient population tested, including the ages of tested children.
Gastric fluid is often sampled in children unable to produce sputum. In samples from children with microbiologically-confirmed TB Xpert performed well on gastric aspirates, with a similar relative sensitivity to liquid culture as for sputum samples (81.8% for gastric fluid vs. 80.3% for sputum). The higher sensitivity of both MODS and Xpert on sputum for clinically confirmed cases is likely due to confounding by age: gastric aspirates were collected from younger children less likely to yield bacilli. However, relatively few gastric aspirates were available for analysis in this study (n=20), and more extensive evaluations of optimal sampling strategies in children, including string test and nasopharageal aspiration techniques, are required to maximize diagnostic efficacy [23].
Test failure (invalid and error reports) has important cost implications. The Xpert failure rate was acceptably low in this study (2.7%) and comparable with reports from demonstration sites [24].
Although MDR TB in children remains relatively rare, the consequences of delayed diagnosis are grave. The ability to rapidly detect RIF resistance in this vulnerable population is a major advance, but care must be taken, in light of the low positive predictive value in populations with a low MDR prevalence [25,26], to obtain confirmatory testing. The small number (n=2) of RIF resistant cases detected in this study does not allow robust conclusions to be drawn.
This study was performed on retrospective stored samples and not all the original samples were available for analysis which may have introduced sampling bias. It is possible that sample storage has reduced the sensitivity of Xpert in comparison with the MODS assay, which was performed on fresh samples. However, the results agree with previous reports that Xpert is a excellent test for diagnosis of pediatric TB and can be applied to both sputum and gastric aspirate samples. Xpert should be evaluated more widely using alternative sampling strategies for diagnosis of pediatric TB.