- Open Open Peer Review
Rational use of Xpert testing in patients with presumptive TB: clinicians should be encouraged to use the test-treat threshold
BMC Infectious Diseasesvolume 17, Article number: 674 (2017)
A recently published Ugandan study on tuberculosis (TB) diagnosis in HIV-positive patients with presumptive smear-negative TB, which showed that out of 90 patients who started TB treatment, 20% (18/90) had a positive Xpert MTB/RIF (Xpert) test, 24% (22/90) had a negative Xpert test, and 56% (50/90) were started without Xpert testing. Although Xpert testing was available, clinicians did not use it systematically. Here we aim to show more objectively the process of clinical decision-making.
First, we estimated that pre-test probability of TB, or the prevalence of TB in smear-negative HIV infected patients with signs of presumptive TB in Uganda, was 17%. Second, we argue that the treatment threshold, the probability of disease at which the utility of treating and not treating is the same, and above which treatment should be started, should be determined. In Uganda, the treatment threshold was not yet formally established. In Rwanda, the calculated treatment threshold was 12%. Hence, one could argue that the threshold was reached without even considering additional tests. Still, Xpert testing can be useful when the probability of disease is above the treatment threshold, but only when a negative Xpert result can lower the probability of disease enough to cross the treatment threshold. This occurs when the pre-test probability is lower than the test-treat threshold, the probability of disease at which the utility of testing and the utility of treating without testing is the same. We estimated that the test-treatment threshold was 28%. Finally, to show the effect of the presence or absence of arguments on the probability of TB, we use confirming and excluding power, and a log10 odds scale to combine arguments.
If the pre-test probability is above the test-treat threshold, empirical treatment is justified, because even a negative Xpert will not lower the post-test probability below the treatment threshold. However, Xpert testing for the diagnosis of TB should be performed in patients for whom the probability of TB was lower than the test-treat threshold. Especially in resource constrained settings clinicians should be encouraged to take clinical decisions and use scarce resources rationally.
With great interest we read the article by Hermans et al. “Treatment decisions and mortality in HIV-positive presumptive smear-negative TB in the Xpert MTB/RIF era: a cohort study” . The authors assessed if clinicians based their clinical decision-making on available Xpert MTB/RIF (Xpert) testing to diagnose tuberculosis (TB) in HIV-positive smear-negative presumptive TB patients. They found that less than half of those diagnosed with smear-negative TB were tested with Xpert MTB/RIF. Hermans et al. concluded that Xpert usage was lower than expected in a setting where Xpert testing is easy accessible. Out of 90 patients who started TB treatment, 20% (18/90) had a positive Xpert test, 24% (22/90) had a negative Xpert test, and 56% (50/90) were started without Xpert testing. The authors concluded that clinicians used Xpert testing as a rule-in test, not that much as a rule-out test, and attribute this to its lower sensitivity in smear-negative HIV-positive patients.
What does this mean, and can this statement be illustrated in a more objective and quantitative manner? Although clinicians are not well acquainted with measures of diagnostic test performance , they want to know how much the probability of having a disease is affected by the presence or absence of a clinical sign or a positive or negative test result, like Xpert. Therefore, to show more objectively the process of clinical decision-making, the following important questions require an answer:
What is the pre-test probability of TB, in smear-negative HIV infected patients with signs of presumptive TB, in this setting?
At which probability should TB treatment be started (treatment threshold)?
At which probability should treatment be started without additional testing (test-treat threshold)?
What is the confirming and excluding power of having a positive or negative Xpert result?
What is the confirming and excluding power of other clinical arguments?
Pre-test probability of TB
The pre-test probability of TB is the prevalence of TB in this subgroup of smear-negative HIV-positive patients. Given that 19 out of 171 individuals tested positive with Xpert, and the 61% sensitivity and 99% specificity of Xpert to diagnose TB in HIV infected persons with a negative smear , we can estimate that about 18, 1, 11, and 141 individuals had respectively a true positive, false positive, a false negative, and true negative Xpert result (Fig. 1). Thus, the pre-test probability was probably around 17% (29/171) among those tested with Xpert. Given that TB symptoms, cough, fever, night sweats, weight loss, anorexia, and chest pain did not differ between the two groups, the pre-test probability was likely similar in both groups (those tested and those not tested with Xpert).
The pre-test probability together with the test result influence the post-test-probability of disease.
Another important concept in clinical decision making is the concept of treatment threshold, first developed by Pauker and Kassirer . At what probability of disease is there equipoise between treating and not treating, where is the utility of treating or not treating the same? How many patients are you willing to treat while they do not have the disease (false positives) in order to save 1 who has the disease (true positive). Regardless of the tests available, the minimal level of probability of disease required to treat a patient is called the treatment threshold. This is only influenced by treatment and disease factors. The treatment threshold will be lower when:
The target disease is severe and/or contagious, like TB.
The treatment is effective, well tolerated, available, affordable, and acceptable to patients.
Given the relatively high net benefit of treating those who truly have the disease compared to the relatively low risk of treatment in patients with a false positive test result, the threshold is pushed to a lower probability level. In Rwanda, the calculated treatment threshold, based on expected utilities and regret, was 12% .
This does not imply that 88% of those started on TB treatment have no tuberculosis. The probability of disease follows a bimodal distribution . In fact, most patients will have either a low or a high probability of disease, only few will be around the treatment threshold . In Uganda, the treatment threshold has not been calculated, but it is unlikely to differ much from the one calculated in Rwanda. If anything, the treatment threshold would be even lower, because of the fact that all patients in the Uganda study are HIV-positive.
Interestingly, Hermans et al. inform the reader that clinicians had to indicate, prior to ordering an Xpert test, if they already had enough arguments to start TB treatment. In other words, they indicated if they had surpassed the treatment threshold . Given the 17% pre-test probability, empirical TB treatment seems justified in this patient population . One could argue that the threshold has been reached without even considering additional tests. When then is it justified to still test with Xpert, even when the probability of disease is above the treatment threshold?
When you have a good diagnostic test you may consider three options: 1) to treat without testing, 2) to not treat and not test, and 3) to do a test first. This is the threshold approach to clinical decision making . The test-treat threshold is the probability of disease at which the utility of testing and the utility of treating without testing is the same. A final, additional test is only useful when it can raise or lower the probability of disease enough to cross a treatment threshold. When a negative test result will not bring the probability of disease below the treatment threshold, treatment is started without further testing. This occurs when the pre-test probability is higher than the test-treat threshold (Fig. 2) [7, 9].
The test-treat threshold is determined not only by disease and treatment factors (treatment threshold) but also by the excluding power of a negative test. We estimated that the test-treat threshold was 28% (Fig. 2).
Confirming and excluding power of an argument
The confirming and excluding power of an argument are derived from likelihood ratios (LR), which are often encountered in papers on diagnostic accuracy. Unfortunately, LR are interpreted in different directions: positive LR range between 1 and infinity whereas negative LR range between 1 and 0 . The further from 1, the more the argument has an effect on the probability of the disease . Thus, an argument with a negative LR of 0.01 has more power than an argument with a negative LR of 0.1, which seems counterintuitive . More logic and easier to understand are confirming and excluding power, which are expressed as an absolute number. The higher the numeric value, the higher the power of the argument to confirm or exclude the disease.
The confirming power is the same as a positive LR, it expresses how many times more likely is a positive test result in a diseased compared to a non-diseased person, and is calculated as the sensitivity divided by (1 – specificity).
Excluding power is the inverse of a negative LR, it expresses how many times more likely is a negative test result in a non-diseased compared to a diseased person, and is calculated as the specificity divided by (1 – sensitivity).
Values from 2 to 5 result in small changes in the post-test probability of the disease, from 5 to 10 moderate changes, and above 10 large changes .
Confirming and excluding power of Xpert results
Indeed, an Xpert result showing “TB detected” has a confirming power of 61, and is a very strong argument in favour of TB diagnosis. Conversely, an Xpert result showing “TB not detected” has an excluding power of 3, and can be considered as a weak argument against diagnosing TB . To use Xpert as a rule-in test, but not a rule-out test, is thus well justified in this study population.
Combining confirming and excluding power of other clinical arguments
Most TB diagnoses were made without Xpert testing. The article doesn’t provide a lot of data on the arguments that informed the clinical decisions made . Still, the authors mention that the chest X-rays (CXR) showed abnormalities in respectively 69% and 54% of patients with and without an Xpert result. In addition, signs of extrapulmonary TB were present in respectively 17% and 50% of patients with and without an Xpert result .
From Cain et al. we learn that the presence of an abnormal CXR has a weak confirming power of 4 (positive LR = 4.42), and a weak excluding power of 2 (negative LR = 0.41) . Lymphadenopathy (any location), which can be interpreted as a clinical sign of extrapulmonary TB in patients with presumptive TB, has a weak confirming power of 3 (positive LR = 3.19), and an excluding power of 1 (negative LR = 0.77) . Values close to 1 are useless for clinical decision-making .
To show the effect of the presence or absence of these arguments on the probability of TB, and how arguments can be combined, we use a log10 odds scale. The mathematical background of this approach has been published elsewhere . Moreover, to simplify the use of the scale, the confirming and excluding power of arguments is categorized into very strong, strong, good, weak, or useless (Table 1) . Figure 3 shows how the probability of TB moves upward or downward on a log10 odds scale, as arguments are present or absent in two hypothetical patients.
Test or treat?
As illustrated in Fig. 2, if the pre-test probability is above the test-treat threshold, empirical treatment is justified, because even a negative Xpert will not lower the post-test probability below the treatment threshold.
Based on the reasoning shown in the paragraphs here above, Xpert testing for the diagnosis of TB should only have been performed in patients for whom the probability of TB was lower than the test-treat threshold [7,8,9] (Fig. 4).
In case Xpert is done anyway for reasons of diagnosing rifampicin resistance, or because guidelines prescribe it for people living with HIV, a negative result at such high post-test probabilities should not make the clinician doubt to treat the patient anyway for TB .
In settings with a lower pre-test probability of having TB, as is often the case in high-income countries, and where there are no financial barriers in using Xpert testing, testing is always justified in patients with presumptive TB. Whether a negative Xpert allows to exclude TB as a diagnosis depends on the treatment threshold established in these settings, which may differ slightly from resource-poor settings.
In conclusion, this example illustrates well the process of clinical decision-making. Especially in resource constrained settings clinicians should be encouraged to take clinical decisions and use scarce resources rationally. Moreover, clinical decision-making is nor linear or static, rather an iterative process. Every follow-up consultation the clinician will re-assess the patient. New signs may emerge, and thus alter the clinical decision-making process. The described structured and objective approach may not be feasible to conduct during every clinical consultation, however, the concepts of confirming and excluding power and test-treat threshold may provide structure to the clinician when ruling in or out a diagnosis.
Hermans SM, Babirye JA, Mbabazi O, Kakooza F, Colebunders R, Castelnuovo B, et al. Treatment decisions and mortality in HIV-positive presumptive smear-negative TB in the Xpert™ MTB/RIF era: a cohort study. BMC Infect Dis. 2017;17(1):433.
Whiting PF, Davenport C, Jameson C, Burke M, Sterne JA, Hyde C, et al. How well do health professionals interpret diagnostic information? A systematic review. BMJ Open. 2015;5(7):e008155.
Steingart KR, Schiller I, Horne DJ, Pai M, Boehme CC, Dendukuri N. Xpert® MTB/RIF assay for pulmonary tuberculosis and rifampicin resistance in adults. Cochrane Database Syst Rev. 2014;1:CD009593.
Pauker SG, Kassirer JP. Therapeutic decision making: a cost-benefit analysis. N Engl J Med. 1975;293(5):229–34.
Basinga P, Moreira J, Bisoffi Z, Bisig B, Van den Ende J. Why are clinicians reluctant to treat smear-negative tuberculosis? An inquiry about treatment thresholds in Rwanda. Med Decis Mak. 2007;27(1):53–60.
Wilson JM. Current trends and problems in health screening. J Clin Pathol. 1973;26(8):555–63.
Hunink MGM. Decision making in health and medicine: integrating evidence and values. 2nd ed. Cambridge: Cambridge University Press; 2014.
Boyles TH. Why do clinical trials of Xpert® MTB/RIF fail to show an effect on patient relevant outcomes? Int J Tuberc Lung Dis. 2017;21(3):249–50.
Pauker SG, Kassirer JP. The threshold approach to clinical decision making. N Engl J Med. 1980;302(20):1109–17.
Van den Ende J, Moreira J, Basinga P, Bisoffi Z. The trouble with likelihood ratios. Lancet. 2005;366(9485):548.
Grimes DA, Schulz KF. Refining clinical diagnosis with likelihood ratios. Lancet. 2005;365(9469):1500–5.
Cain KP, McCarthy KD, Heilig CM, Monkongdee P, Tasaneeyapan T, Kanara N, et al. An algorithm for tuberculosis screening and diagnosis in people with HIV. N Engl J Med. 2010;362(8):707–16.
Van den Ende J, Bisoffi Z, Van Puymbroek H, Van der Stuyft P, Van Gompel A, Derese A, et al. Bridging the gap between clinical practice and diagnostic clinical epidemiology: pilot experiences with a didactic model based on a logarithmic scale. J Eval Clin Pract. 2007;13(3):374–80.
Data sharing statement
Availability of data and materials
This research received no specific grant from any funding agency in the public, commercial, or non-profit sectors.
Ethics approval and consent to participate
Consent for publication
The authors declared that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.