The realistic performance achievable with mycobacterial automated culture systems in high and low prevalence settings
© van Kampen et al; licensee BioMed Central Ltd. 2010
Received: 1 February 2010
Accepted: 12 April 2010
Published: 12 April 2010
Diagnostic tests are generally used in situations with similar pre-test probability of disease to where they were developed. When these tests are applied in situations with very different pre-test probabilities of disease, it is informative to model the likely implications of known characteristics of test performance in the new situation. This is the case for automated Mycobacterium tuberculosis (MTB) liquid culture systems for tuberculosis case detection which were developed and are widely used in low burden settings but are only beginning to be applied on a large scale in high burden settings.
Here we model the performance of MTB liquid culture systems in high and low tuberculosis (TB) prevalence settings using detailed published data concentrating on the likely frequency of cross-contamination events.
Our model predicts that as the TB prevalence in the suspect population increases there is an exponential increase in the risk of MTB cross-contamination events expected in otherwise negative samples, even with equivalent technical performance of the laboratories. Quality control and strict cross-contamination measures become increasingly critical as the burden of MTB infection among TB suspects increases. Even under optimal conditions the realistically achievable specificity of these systems in high burden settings will likely be significantly below that obtained in low TB burden laboratories.
Liquid culture systems can play a valuable role in TB case detection in laboratories in high burden settings, but laboratory workers, policy makers and clinicians should be aware of the increased risks, independent of laboratory proficiency, of cross-contamination events in high burden settings.
Automated liquid culture for M. tuberculosis (MTB) is widely used in industrialized countries and increases the sensitivity and reduces the time required for culture based tuberculosis (TB) diagnosis. The use of this highly sensitive technique increases the importance of robust protocols and quality control to prevent and if this occurs detect laboratory cross-contamination. Protocols and quality control systems have been developed over the past years that are adequate to ensure excellent performance in settings with a low burden of TB. With the recent trends in TB diagnostics for high burden countries moving towards more modern technologies and scale-up of laboratories for the implementation of new methods, we and others in the field have expressed concern about the feasibility and viability of these changes in certain locations [1, 2]. Laboratories in high burden TB areas that perform diagnosis based on conventional microscopy are now for example recommended to move towards innovative liquid culturing systems for case detection and molecular line probe assays for drug susceptibility testing. These new technologies are more sensitive and rapid, but the concern is that they are also more prone to laboratory cross-contamination potentially resulting in misdiagnosis. This is especially problematic when these tests are performed in laboratories with inadequate infrastructure and maintenance, or laboratories that lack the possibility of confirmatory tests. This is often the case in countries with a high TB burden. The consequences of a high contamination risk are numerous, but essentially results in bacterial culture results from true negative samples becoming positive. This issue forms the focus of our study. As the treatment of TB patients is long and complex with the potential for side effects, a false-positive MTB test is not trivial and places a significant burden on the TB suspect, the suspect's family and the health system in general [3, 4].
Many studies have investigated the risks, causes and consequences of laboratory cross-contamination both in low- and high burden TB settings. However, the effect of the prevalence of positive MTB samples in the laboratory - which is strongly associated with the TB prevalence in the area - is generally not explicitly considered with respect to the absolute risk of cross-contamination of a true negative sample. The aim of this study is to model how cross-contamination risk, in laboratories with different performance, is influenced by the rate of samples positive for MTB processed in the laboratory for liquid culture.
First, a mathematical model is used to calculate the risk of a negative sample being tested false-positive based on various assumptions. It is assumed that the false positive rate (FPR) and false negative rate (FNR) are constant within a laboratory and depend only on the laboratory's performance. The variable factor under investigation is the prevalence of positive MTB samples (PSP) in the tested batch of samples, which influences the chance that a cross-contamination event will result in sample contamination with mycobacteria.
Diagnostic 2 × 2 Table
False positive rate (number of false positive samples divided by total number of positive samples) is F = FP/p
False negative rate (number of false negative samples divided by total number of negative sample) is G = FN/q
In reality of course there will be a small portion of false-negatives, meaning that we slightly underestimate R with our formula.
We used published study data from real-life laboratory settings to explore the implications of this formula. A study on culture cross-contamination rates in Dutch laboratories provides data representing situations with a low prevalence of TB and low levels of cross contamination. In that study, all culture results of the TB national reference laboratory (NRL) were investigated over a seven-year period (1993 until 2000). The Dutch laboratories sent on average 1570 culture samples per year to the NRL, of which 21 were tested culture-positive by the NRL, leading to a mean PSP of 21/1570 = 0.0134 = 1.34%. Since the formula reflects rates, the number of positive and negative samples (p and q) can be replaced by the rate of positive and negative samples compared to the total number of samples in a batch (PSP and 1-PSP). The NRL also typed all positive TB cultures using DNA fingerprinting. It was estimated, that the FPR in this time period was 2.4% on average, ranging from 3.9% at the start to 1.1% at the end of the investigation. It should be noted, that the very low FPR of 1.1% (F = 0.011) was reached after cross-contamination investigations had been performed in the Dutch laboratories and advice had been given about how to minimize the FPR over a number of years. We assume this extremely low FPR to represent the maximum practically achievable specificity of the test under ideal conditions.
With a constant FPR and increasing or decreasing the prevalence of positive TB samples in the South African laboratory, the relating contamination risks are depicted in figure 1 (solid line). Again, an exponential relation is seen between PSP and contamination risk.
Considering that a PSP of up to 55% does occur in high burden labs such as the South African laboratory, the part of the graph that is considered to represent a realistic situation is enlarged in figure 1 (lower graph). It is clear that when the FPR in a laboratory is kept constant, the risk of a sample getting cross-contaminated rises exponentially when the positive sample rate in the tested batch rises. Critically with a slightly raised FPR and high PSP this effect can quickly become a serious issue.
When applying diagnostic tests developed for use in situations with a very different pre-test probability of disease, it is informative to model the likely implications on specific aspects of test performance in the new situation. In this report we model the likely performance of M. tuberculosis automated liquid culture systems in high-burden countries.
Our calculations of the expected performance of liquid culture in different settings show that a laboratory with optimized procedures that achieves a low FPR (e.g. 1.1% - as for the Dutch laboratories) constitutes a low risk for a MTB negative TB suspect sample becoming contaminated, even when a high proportion of true positives are tested. When a contamination risk of 2.5% is taken as cut-off value (meaning that a higher contamination risk for a true negative sample is unacceptable, equivalent to a greater than 1 in 40 chance of a negative sample being falsely called positive), the PSP in a laboratory with a FPR of 1.1% due to cross-contamination needs to increase to over 60% in order for the contamination risk to rise above the cut-off value. Fortunately, a PSP of over 60% is unlikely in most countries. However, if the FPR is 7.3% - as for the South African laboratory when liquid culture was initially introduced -, the 2.5% cut-off value for contamination risk of a negative sample is reached when the PSP gets to 26%. With a FPR of 2.1% the PSP must reach 55% before the contamination risk of a true negative sample reaches 2.5%.
Over the previous years detailed and careful work has accurately defined the primary concern with automated liquid culture systems for the detection of M. tuberculosis infection. Namely, as a consequence of the comparatively rapid growth achieved and their excellent sensitivity they are more sensitive to sample cross-contamination than previous methods. We focused on this issue in this report. Once this problem is recognized the impact can be assessed and minimized. This results in a marked increase in performance in the years following adoption of the methods as well as the need for sufficient throughput and investment to maintain laboratory skills. Of course even in the best laboratories some level of cross-contamination is inevitable and is this report we aimed to be quite conservative regarding the likely cross contamination rates. In industrialized countries with functioning quality systems and active reference laboratory support this level appears to be around 1% of all positive samples .
The cross-contamination risk for true negative samples is not only dependent on a laboratories performance (mostly measured by FPR), but is also strongly influenced by the MTB PSP in the laboratory. Unfortunately, as our calculations demonstrate, the implications for even relatively low levels of cross-contamination are much more serious in locations with high TB prevalence in suspects. These laboratories are ironically at present often the least equipped to fully implement all the required protocols. These precautions include: separating the processing of non-sterile specimens and specimen with a high risk of contamination (e.g. specimen of known positive cultures, proficiency-test samples or contaminated culture vials); using individual aliquots of buffer solution; uncapping only one tube at a time; leaving samples at rest for five minutes after centrifugation or mixing; decreasing the maximum number of samples processed in one batch; re-enforcing hygiene protocols and clean laboratory practice among staff; and raising awareness and cautiousness of the possibility of false-positive results in general [7–9]. In order to monitor the performance of TB culture based diagnostic programs it is also useful to have the facility to genetically type a proportion of isolated strains, a service which is also frequently unavailable were it is most needed .
It has to be noted that in this study only the risk of cross-contamination leading to false-positive case detection is discussed. Like in most studies, the research articles used here focus on calculating the false-positive culture rate, i.e. the proportion of positive samples that is false-positive for the presence of MTB. However, they do not take into account the possible risk of a positive MTB culture sample being cross-contaminated with another positive sample, or the risk of a positive sample being contaminated with a (multi-)drug resistant MTB sample. The first event could potentially alter the Mycobacterium tuberculosis family type and lead to errors in determining epidemiological patterns. The latter could lead to a misdiagnosis of drug susceptibility of a sample and consequently to a patient receiving an incorrect treatment regimen. Thus, laboratory cross-contamination can also lead to erroneous detection of (multi-)drug resistance outbreaks or other epidemiological outbreaks. Since this study does not measure the risks of these other cross-contamination events, nor the effect that MTB positive sample rate might have on these risks, it is suggested that future studies also focus on these factors.
It can be concluded that even if future global laboratory interventions are able to optimize the culture procedures leading to similar low false-positive culture rates in all laboratories around the world, the mathematical model described in this study predicts that the risk of contamination of negative cultures will inevitably remain higher in areas with a higher TB prevalence. It is likely that alternative liquid culture systems will have similar characteristics. Therefore, it is of critical importance that clinicians and policy makers involved in TB diagnostics remain aware of the possibility of laboratory cross-contamination, and take into account the regional differences when interpreting results and potential of these highly sensitive culture methods.
false-positive culture rate
false-negative culture rate
national reference laboratory
positive sample prevalence
We would like to thank our colleague Dr. Mirjam Bakker for her valuable help with the development of the mathematical model.
- Anthony RM, Cobelens FG, Gebhard A, Klatser PR, Lumb R, Rüsch-Gerdes S, van Soolingen D: Liquid culture for Mycobacterium tuberculosis: proceed but with caution. Int J Tuberc Lung Dis. 2009, 13: 1051-1053.PubMedGoogle Scholar
- Urbanczik R, Rieder HL: Scaling up tuberculosis culture services: a precautionary note. Int J Tuberc Lung Dis. 2009, 13: 799-800.PubMedGoogle Scholar
- Martinez M, Garcia de Viedma D, Alonso M, Adrés S, Bouza E, Cabezas T, Cabeza I, Reyes A, Sánchez-Yebra W, Rodríguez M, Sánchez MI, Rogado MC, Fernández R, Peñafiel T, Martínez J, Barroso P, Lucerna MA, Diez LF, Gutiérrez C: Impact of laboratory cross-contamination on molecular epidemiology studies of tuberculosis. J Clin Microbiol. 2006, 44: 2967-2969. 10.1128/JCM.00754-06.View ArticlePubMedPubMed CentralGoogle Scholar
- Cronin W, Rodriguez E, Valway S, Bur S, Hooper N, Smithwick R, Butler WR, Dwyer D: Pseudo-outbreak of tuberculosis in an acute-care general hospital: epidemiology and clinical implications. Infect Control Hosp Epidemiol. 1998, 19: 345-347. 10.1086/647827.View ArticlePubMedGoogle Scholar
- De Boer AS, Blommerde B, de Haas PE, Sebek MM, Lambregts-van Weezenbeek KS, Dessens M, van Soolingen D: False-positive Mycobacterium tuberculosis cultures in 44 laboratories in the Netherlands (1993 to 2000): Incidence, risk factors, and consequences. J Clin Microbiol. 2002, 40: 4004-4009. 10.1128/JCM.40.11.4004-4009.2002.View ArticlePubMedPubMed CentralGoogle Scholar
- Carroll NM, Richardson M, Engelke E, de Kock M, Lombard C, van Helden PD: Reduction of the rate of false-positive cultures of Mycobacterium tuberculosis in a laboratory with a high culture positivity rate. Clin Chem Lab Med. 2002, 40: 888-892. 10.1515/CCLM.2002.157.View ArticlePubMedGoogle Scholar
- Poynten M, Andresen DN, Gottlieb T: Laboratory cross-contamination of Mycobacterium tuberculosis: an investigation and analysis of causes and consequences. Intern Med J. 2002, 52: 512-519. 10.1046/j.1445-5994.2002.00271.x.View ArticleGoogle Scholar
- Burman WJ, Reves RR: Review of false-positive cultures for Mycobacterium tuberculosis and recommendations for avoiding unnecessary treatment. Clin Infect Dis. 2000, 31: 1390-1395. 10.1086/317504.View ArticlePubMedGoogle Scholar
- Breese PE, Burman WJ, Hildred M, Stone B, Wilson ML, Yang Z, Cave MD: The effect of changes in laboratory practices on the rate of false-positive cultures for Mycobacterium tuberculosis. Arch Pathol Lab Med. 2001, 125 (9): 1213-1216.PubMedGoogle Scholar
- Spuy Van der GD, Warren RM, van Helden PD: The role of molecular epidemiology in low-income, high-burden countries. Int J Tuberc Lung Dis. 2009, 13: 419-420.PubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2334/10/93/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.