- Study protocol
- Open Access
Implementation of a pragmatic, stepped-wedge cluster randomized trial to evaluate impact of Botswana’s Xpert MTB/RIF diagnostic algorithm on TB diagnostic sensitivity and early antiretroviral therapy mortality
BMC Infectious Diseases volume 16, Article number: 606 (2016)
In 2012, as a pilot for Botswana’s national Xpert MTB/RIF (Xpert) rollout plans, intensified tuberculosis (TB) case finding (ICF) activities were strengthened at 22 HIV treatment clinics prior to phased activation of 13 Xpert instruments. Together, the strengthened ICF intervention and Xpert activation are referred to as the “Xpert package”.
The evaluation, called the Xpert Package Rollout Evaluation using a Stepped-wedge design (XPRES), has two key objectives: (1) to compare sensitivity of microscopy-based and Xpert-based pulmonary TB diagnostic algorithms in diagnosing sputum culture-positive TB; and (2) to evaluate impact of the “Xpert package” on all-cause, 6-month, adult antiretroviral therapy (ART) mortality. A pragmatic, stepped-wedge cluster-randomized trial design was chosen. The design involves enrollment of three cohorts: (1) cohort R, a retrospective cohort of all study clinic ART enrollees in the 24 months before study initiation (July 31, 2012); (2) cohort A, a prospective cohort of all consenting patients presenting to study clinics after study initiation, who received the ICF intervention and the microscopy-based TB diagnostic algorithm; and (3) cohort B, a prospective cohort of all consenting patients presenting to study clinics after Xpert activation, who received the ICF intervention and the Xpert-based TB diagnostic algorithm. TB diagnostic sensitivity will be compared between TB culture-positive enrollees in cohorts A and B. All-cause, 6-month ART-mortality will be compared between cohorts R and B. With anticipated cohort R, A, and B sample sizes of about 10,131, 1,878, and 4,258, respectively, the study is estimated to have >80 % power to detect differences in pre-versus post-Xpert TB diagnostic sensitivity if pre-Xpert sensitivity is ≤52.5 % and post-Xpert sensitivity ≥82.5 %, and >80 % power to detect a 40 % reduction in all-cause, 6-month, ART mortality between cohorts R and B if cohort R mortality is ≥13/100 person-years.
Only one small previous trial (N = 424) among ART enrolees in Zimbabwe evaluated, in a secondary analysis, Xpert impact on all-cause 6-month ART mortality. No mortality impact was observed. This Botswana trial, with its larger sample size and powered specifically to detect differences in all-cause 6-month ART mortality, remains well-positioned to contribute understanding of Xpert impact.
Retrospectively registered at ClinicalTrials.gov: NCT02538952.
In Botswana, as in the rest of sub-Saharan Africa, undiagnosed tuberculosis (TB) or TB diagnosed late in the course of disease is thought to be the most common cause of death among persons living with HIV (PLHIV), whether they are receiving antiretroviral therapy (ART) or not, with TB accounting for about 40 % of deaths according to a recent meta-analysis of pathological autopsy studies . Although antiretroviral therapy (ART) reduces risk of all-cause mortality among PLHIV, early mortality in the first 3–6 months after ART initiation remains high in sub-Saharan Africa and is commonly due to undiagnosed TB or TB diagnosed late [2–4].
Reasons for failure to diagnose TB early among PLHIV can be categorized as patient-related or health facility-related. Patient-related reasons include: (1) failure to present to a health facility when symptoms arise, which may be due to poor access to health care or cultural norms that delay health-seeking behavior , and (2) absence of TB symptoms among late presenters with advanced immune suppression because TB symptoms are dependent on both bacillary burden and immune response .
Healthcare facility-related reasons for missed or late TB diagnoses among PLHIV include: (1) failure of healthcare workers (HCW) to screen for TB symptoms [7–9]), (2) failure of HCWs to request sputum or other diagnostic samples from symptomatic patients , (3) inability to collect high quality sputa or other appropriate diagnostic samples from symptomatic patients , (4) insensitive TB diagnostics, with smear microscopy alone having a sensitivity of about 45 % in diagnosing culture-positive disease among PLHIV , (5) inability to diagnose drug-resistant TB timeously , and (6) long turn-around times for some TB diagnostic tests or failure to return results to clinicians and patients .
In 2009, the commercial release of the Xpert MTB/RIF assay (Xpert) for the GeneXpert platform represented an important breakthrough in TB diagnostics. With features including sensitivity of about 79 % in diagnosing culture-positive TB from sputum samples among PLHIV , significantly superior to smear microscopy , ability to detect rifampicin resistance-conferring mutations, capacity to run the test on sputum samples within 100 min after brief sample processing, and minimal laboratory training requirements, Xpert significantly advanced TB diagnostic capability for clinicians managing PLHIV, especially in resource-limited settings (RLS) . However, Xpert on its own, cannot solve all the facility-level challenges to diagnosing TB . Strengthening of the entire TB symptom screening and diagnostic algorithm is needed for Xpert to have maximum impact on patient health outcomes in most RLS .
Therefore, in 2012, the Botswana Ministry of Health (MOH) and the United States Centers for Disease Control and Prevention (CDC), designed a package of intensified TB case finding (ICF) interventions to be rolled out prior to, and in coordination with, the phased activation of 13 Xpert devices in support of 22 HIV care and treatment clinics.
The package of ICF interventions included: (1) ensuring the 22 HIV care and treatment clinics adopted the World Health Organization (WHO)-recommended four-symptom TB screen for adults (>12 years old as defined by the HIV care and treatment program); (2) situating trained TB case-finding nurses in all 22 facilities to implement the screening and diagnostic algorithms; and (3) training TB case-finding nurses and other health facility personnel in both smear-microscopy-based and Xpert-based TB diagnostic algorithms for adults and children. The combination of the ICF interventions and rollout of the Xpert device is referred to as the “Xpert package” in this report.
To evaluate the accuracy of the new MOH-proposed Xpert diagnostic algorithm and also the impact of the whole Xpert package on patient outcomes, a pragmatic, stepped-wedge cluster-randomized trial (CRT), referred to as the Xpert Package Rollout Evaluation using a Stepped-wedge design (XPRES), was initiated. In this paper, the protocol-specified key study objectives, design rationale, sample size, key procedures, and analytic approaches are described. In addition, the evolution of power estimates over time as real-time study enrollment numbers became available, and key amendments to study procedures, which were needed to adapt to operational challenges, are described.
The first objective of the evaluation is to determine whether the new MOH-recommended Xpert-based pulmonary TB diagnostic algorithm (including Xpert testing of sputum samples for all patients screening positive (i.e., presumptive TB patients) and chest x-ray for Xpert-negative presumptive TB patients) is more sensitive than the pre-Xpert smear-microscopy-based algorithm (smear microscopy and chest x-ray for smear-negative presumptive TB patients) in diagnosing culture-positive TB disease among adult PLHIV. Although it is expected that the Xpert-based TB diagnostic algorithm will be both more sensitive and more specific than the pre-Xpert algorithm, superiority of the Xpert algorithm has not yet been demonstrated in Botswana and thorough evaluation of the accuracy of the new diagnostic algorithm is important to guide future investments [18, 19].
The second objective is to evaluate the impact of the whole Xpert package on all-cause mortality during the first six months of ART, among adult PLHIV. With an estimated 40 % of early ART deaths due to undiagnosed TB or TB diagnosed late, the Xpert package could conceivably reduce all-cause, 6-month ART mortality by ensuring: (1) that all ART enrollees are appropriately screened for TB symptoms before and during ART, and (2) that presumptive TB patients have access to a sensitive TB test (Xpert) and early TB treatment where warranted . Only one small trial (N = 424) in Zimbabwe has previously aimed to examine the impact of Xpert-versus microscopy-based TB diagnostic algorithms on 6-month ART mortality ; no difference in early ART mortality was noted between study arms, however, the small sample size and high rates of empiric TB treatment in both study arms limit study findings.
Study design rationale
A pragmatic stepped-wedge CRT design was chosen because: (1) Xpert device activation was most feasibly achieved for an entire district TB laboratory, which often served more than one health facility; this fact made an individual randomized controlled trial (RCT) design less desirable , (2) according to WHO guidance  and MOH guidelines , the Xpert device was expected to be beneficial for both patients and providers, and therefore it was considered ethically sub-optimal to implement a parallel group CRT, where certain district TB labs and their associated clinics were denied access to Xpert for an extended period of time [12, 24], (3) the phased rollout of Xpert provided logistical advantages, because it meant that a single site activation team, in charge of training and activation of the Xpert device, could sequentially initiate all study sites , (4) the need for only a single site activation team reduced projected study cost, (5) program managers and funders were interested in assessing accuracy of the Xpert diagnostic algorithm in a real-world environment rather than trying to assess accuracy in a tightly controlled research environment, with limited external validity , (6) in a real-world setting, the sequential rollout of an intervention allows lessons learned during earlier steps to be applied during later steps, and (7) a stepped-wedge design provides analysis options that allow for the control of trends over time [26, 27].
Study design description
Figure 1 summarizes the study design. This step-wedge design involved enrollment of three cohorts: (1) retrospective cohort (R), shaded in red in Fig. 1, (2) prospective cohort A (enrolled pre-Xpert device rollout), shaded in yellow in Fig. 1, and (3) prospective cohort B (enrolled post-Xpert device rollout), shaded in green in Fig. 1. For cohort R, all patients who initiated ART at one of the 22 HIV clinics for the first time in the 24 months before study start (i.e., before July 31, 2012), were eligible for enrollment. For cohort A, all patients who attended one of the 22 HIV clinics for the first time after study start (July 31, 2012), but before Xpert device rollout, were eligible for enrollment. For cohort B, all patients who attend one of the 22 HIV clinics for the first time after Xpert device rollout were eligible.
To answer the first primary study question, sensitivity of the pre-Xpert TB diagnostic algorithm in prospective cohort A will be compared with the post-Xpert algorithm sensitivity in prospective cohort B. Figure 2 describes the differences in TB diagnostic algorithms between cohorts A and B and how sensitivity proportions will be determined.
To meet the second key study objective (comparison of pre- versus post-Xpert package all-cause ART mortality), all-cause 6-month ART mortality rates will be compared between the retrospective cohort (cohort R) and the post-Xpert prospective cohort (cohort B). Since the cohorts being compared (cohorts R and B) do not overlap in a phased manner that would allow for controlling for secular trends according to analytic approaches recommended by Moulton et al  or Hussey & Hughes , this analysis approach is best characterized as a before and after comparison. However, secondary analyses, comparing 6-month ART mortality, and other ART outcomes, between cohorts A and B will make use of the stepped-wedge portion of the trial and analytic approaches recommended by Moulton et al  and Hussey & Hughes  to control for secular trends.
As described above, Fig. 2 illustrates the differences between the microscopy-based algorithm used in cohort A (pre-Xpert device rollout), and the Xpert-based algorithm used in cohort B (post - Xpert device rollout).
For the second key question, comparing all-cause 6-month ART mortality in cohort R with cohort B, Table 1 summarizes the differences between cohort R and B in terms of TB case finding and patient management activities. Notably, in addition to implementing ICF activities, study nurses were responsible for tracing prospectively enrolled patients (patients in cohorts A and B) who were ≥1 day late for a clinic appointment through up to five telephone calls and two home visits to return patients to HIV care. Indicators measuring compliance with all interventions, including implementation of the appropriate TB diagnostic algorithm and the tracing intervention, will help inform discussions of causal pathways during analysis of intervention impact on 6-month ART mortality.
The main reason for selecting the 22 facilities in Table 2 is that they are considered by study investigators and MOH to be representative of facilities in Botswana in terms of TB case finding capacity and ART service delivery. Study facilities consist of five district hospitals and 17 primary healthcare clinics (PHCs). Other advantages of choosing these facilities are: (1) on average they had anticipated high patient enrollment rates (at 33 patients/month/clinic), which helped meet the desired study power (see below), (2) one clinic (Gantsi, in Western Botswana) is estimated to have high prevalence of multi-drug resistant TB among HIV clinic enrollees and so could benefit from early rollout of the Xpert device per WHO recommendations , and (3) all 22 clinics had at least one year’s experience in providing ART services by the time of study start. Table 2 summarizes the study facilities chosen for XPRES.
For the retrospective cohort, all patients starting ART in the 24 months before study start, except for prisoners, were eligible for chart abstraction to estimate all-cause ART mortality rates. For the prospective cohorts (A and B), all patients, who met consent requirements, registering for HIV care for the first time at the facility in the 19 months after study start, were eligible for enrollment, except for prisoners. Prisoners were excluded because it would be difficult to obtain comprehensive retrospective cohort data for cohort R, due to frequent unscheduled prisoner movement during incarceration, and difficult to retain prisoners in cohorts A and B for the study’s duration. Children (<12 years of age) were eligible for prospective enrollment, if their assent and guardian’s consent were provided, because secondary study questions aim to estimate Xpert algorithm sensitivity for children and its impact on pediatric ART outcomes.
Because some of the clinics use the same TB diagnostic facility, Xpert activation was simultaneous for these clinic consortiums (Table 2). For scheduling purposes and because clinic staff rotations occurred at the end of calendar months, each step in the stepped-wedge design needed to be equivalent to one calendar month. In addition, because the MOH and partners wanted the 13 Xpert devices to be operational and serving patients in nine months rather than 13 months, there was a need to initiate two Xpert devices during a single step for four of the nine steps. Taking into consideration constraints in Xpert device assignments to clinics (see Table 2 and Fig. 1), there were 9! (362,880) possible permutations of the order of Xpert rollout. The study statistician randomly selected one of these permutations .
Sample size and power—first key objective
Funding availability limited the prospective enrollment duration to 19 months at the 22 study facilities. MOH monitoring data reported an average of 23 new ART-eligible patients enrolling in each facility per month and investigators estimated there were 10 new ART-ineligible patients enrolled per month at each facility (i.e., potentially 33 study-eligible patients/month/clinic). To be conservative with sample size estimates, we assumed that only 70 % of study-eligible patients would agree to prospective enrollment in the study (i.e., 23 study patients/month/clinic), giving anticipated cohort A and B sample sizes of 3,266 and 6,348, respectively (N = 9,614). Since the vast majority of patients (>99 %) at these study clinics were adults (≥12 years old), for the purpose of sample size calculations, we assumed all 9,614 prospective enrollees would be adults.
Based on a published meta-analysis, we estimated that about 49 % of new adult HIV clinic enrollees would screen positive for TB , and that about 33 % of those screening positive would have culture-confirmed TB, giving an overall active TB prevalence among adult study enrollees of about 16.2 % [30–35]. Our literature review suggested true active TB prevalence among adult PLHIV entering HIV care ranged from 7.1 % in Ethiopia  to 31.5 % in South Africa ; since Botswana has a higher TB case notification rate (about 470/100,000 population) than Ethiopia (about 224/100,000 population) but a lower TB case notification rate than South Africa (about 860/100,000 population), our estimate of adult active TB prevalence of 16.2 % at HIV care entry in Botswana was considered reasonable [36–38]. Further literature review suggested that the pre-Xpert, microscopy-based TB diagnostic algorithm sensitivity might be as high as 62.5 % , and that Xpert algorithm sensitivity among symptomatic PLHIV could be about 82.5 % based on data from a recent multi-country Xpert accuracy study .
To estimate power, data were simulated according to the stepped-wedge design, using the beta-binomial model to induce the intra-cluster correlation coefficient. One thousand datasets were simulated and a mixed model appropriate for the stepped-wedge design fit to the data, as described by Hussey and Hughes , that included fixed effects for time and intervention condition (0 for time points before Xpert implementation and 1 afterward), and a random effect for the clinic, to take into account between-clinic variability. For protocol-specified sample sizes (N = 9,614), and assuming culture-positive TB prevalence of 16.2 % at study entry, and pre-versus post-Xpert sensitivity comparisons of 62.5 % vs. 82.5 %, we had 99.6 % power. In multiple simulations, pre-Xpert sensitivity was varied from 55 % to 62.5 % and post-Xpert sensitivity from 70 % to 82.5 % and the study had >80 % power to detect the intervention effect across all simulated scenarios.
After study initiation, monitoring data, prepared by study nurse supervisors during supervision visits, revealed that actual monthly HIV clinic enrollment numbers were lower than expected (about 21 patients/clinic/month instead of 33 patients/clinic/month). In addition, study nurses were only able to enroll about 72 % of study-eligible patients at the clinic, mostly because patients were not willing to wait while the study nurse completed enrollment of other patients, a process which took about 1 h/enrollee. Therefore, prospective enrollment occurred at about 15/month instead of the protocol-specified 23/month. In addition, the proportion of adult patients screening positive for TB at prospective cohort enrollment was lower than expected (24 % instead of 49 %), and the proportion of those screening positive, who were diagnosed with culture-positive TB, was lower than expected (17 % instead of 33 %), giving a much lower culture-positive TB prevalence at enrollment than was originally expected (about 4 % instead of 16 %).
Lower numbers of culture-positive TB cases/clinic/month (1/clinic/month instead of 4/clinic/month) resulted in inability to fit the stepped-wedge model to all simulated datasets. Therefore, the power estimation approach was simplified and Fisher’s Exact Test for comparing two proportions (pre-and post-Xpert) in SAS version 9.2. software (SAS Institute Inc., Cary, NC) was used to estimate study power. Power estimates were then adjusted for the expected design effect to account for intra-cluster correlation. For design effect calculations, an intra-class correlation of 0.05 was assumed . Input estimates for TB prevalence at HIV care enrollment and TB diagnostic sensitivity were varied to understand impact on study power (Fig. 3). As illustrated in Fig. 3, sample size and TB prevalence shortfalls meant we only had about 75.4 % power to detect the protocol-specified difference in sensitivity (62.5 % vs. 82.5 %) if active TB prevalence at enrollment was 4 %. However, we would have >80 % power to detect pre-versus post-Xpert TB diagnostic sensitivities at a culture-positive TB prevalence of 4 % if pre-Xpert sensitivity was ≤52.5 % and post-Xpert sensitivity ≥82.5 % (Fig. 3). Because available National TB Reference Laboratory (NTRL) monitoring data suggested pre-Xpert TB diagnostic sensitivity was ≤52.5 % and Xpert sensitivity ≥82.5 %, the study was considered still well powered to answer the first key study question at quarterly reviews conducted by the study sponsor.
Sample size and power—second key objective
To estimate power for the comparison of all-cause 6-month mortality in the retrospective cohort (cohort R) versus the post-Xpert prospective cohort (cohort B), the approach of Moulton et al, suitable for stepped-wedge trial designs, was chosen because these power estimates were more conservative than those derived from a pre-post sample size calculation . Per this approach, published formulae for the comparison of two rates in an unmatched parallel group CRT  were adapted to the stepped-wedge design as follows:
where Z β is the standard normal deviate corresponding to the upper tail probability of β and β is the probability of a Type II error; c is the number of clusters (study facilities) per arm, where, since this is a stepped-wedge trial involving 22 clinics, 22/2 was used ; r c is the estimated true 6-month ART mortality rate in the pre-intervention control phase (cohort R); r t is the estimated true mortality rate in the post-intervention phase (cohort B); y c is the average number of person-years (PYs) per clinic in the control phase, estimated as the average retrospective cohort size per clinic (552) divided by two since each patient commits 6 months of follow-up time to the analysis; y t is average number of PYs per clinic in the intervention phase, conservatively estimated as the harmonic mean of PYs contributed by each study site in cohort B, again assuming 6 months of follow-up time per participant ; k is the estimated between-cluster coefficient of variation of the true rates in both the control and intervention phases, estimated as 0.2 ; Z a/2 is the standard normal deviate corresponding to the upper tail probability of a/2 where a is the probability of a Type I error.
Since a log-rank test statistic for intervention effect calculated for a simulated stepped-wedge trial (Z SW ) will generally always be lower than the corresponding statistic (Z E ) for a parallel group trial, because allocation ratios of patients to intervention or control status for parallel group trials remain equal while for stepped-wedge trials they are usually unequal, except at the mid-point of the stepped-wedge design, the z-score in the stepped-wedge trial formula (Z β above) was divided by a published estimate of Z E /Z SW (i.e., 1.2) prior to extrapolating the z-score to a power estimate . Similarly, for Type 1 error of 5 %, instead of assuming a Zα/2 of 1.96, an inflated estimate of 2.352 was used, per published precedent .
Prior to study start, available data from Botswana suggested that the documented all-cause early mortality rates in the first 6 months of ART among adults were about 15 deaths per 100 PYs , which was similar to estimates from a meta-analyses of 18 programs in RLS with active tracing programs (14.7/100 PY) . Since Botswana data, and available meta-analyses suggested about 40 % of deaths among PLHIV were due to undiagnosed TB or TB diagnosed late, and given that interrupting ART during the first 6 months of therapy by missing clinic appointments increases mortality risk [42, 43], it was considered reasonable that the Xpert package plus the tracing intervention might reduce mortality by about 40 % [2, 44]. According to protocol-specified sample sizes, the study had >80 % power to detect a difference in 6-month all-cause ART mortality between cohorts R and B of 40 % if cohort R mortality was ≥10/100 PYs (Fig. 4). According to anticipated actual sample sizes, the study has >80 % power to detect a difference in 6-month all-cause ART mortality between cohorts R and B of 40 % if cohort R mortality is ≥13/100 PYs (Fig. 4).
Study procedures related to first key objective
For prospective cohort enrollment, all new HIV clinic enrollees were informed about the study, and if interested, offered the opportunity to enroll following the Institutional Review Board (IRB)-approved consent process. An enrollment questionnaire collected important baseline demographic and clinical information, and a patient locator form was used to document telephone numbers and home addresses for patient retention activities. Adult patients screening positive for TB were asked to provide four sputa, two collected simultaneously the day of enrollment (referred to as “spot” sputa), and two the following day. Of the two sputa provided the second day, one was a morning sputum prepared by the patient soon after waking up, and the other a spot sputum provided upon arrival at the clinic. Additional file 1 shows a poster used by study nurses to inform the patient how to produce a good sputum. The poster also illustrates important infection control precautions (e.g., preparing spot sputa in a well-ventilated, but still private, “cough spot” outside the clinic). If a patient screening positive for TB was unable to spontaneously produce sputa, MOH-recommended sputum-induction procedures were encouraged if there were no contra-indications . For children <12 years old, who were unable to produce sputa spontaneously and were too young for induction (i.e., <5 years old), naso-gastric tube aspirates were recommended per MOH guidelines ; however, very few children <5 were expected to enroll in the study.
Spot sputa numbers one and three were sent to the on-site or peripheral district TB lab for: (1) smear microscopy, and (2) Xpert, if the Xpert device had been activated by that time. Spot sputum 2 and the morning sputum were sent to the NTRL for: (1) fluorochrome acid fast bacilli (AFB) smear microscopy on concentrated specimens, (2) liquid culture in mycobacteria growth indicator tubes, (3) confirmation of any mycobacterial growth as Mycobacterium tuberculosis (MTB) or non-tuberculous mycobacteria through Ziehl-Neelsen staining, blood agar plate, and immunochromatographic assays, (4) line probe assay testing for isoniazid and rifampicin resistance on MTB-positive cultures, and (5) phenotypic culture-based drug susceptibility testing on MTB-positive cultures. All test results were returned to study nurses with recommended maximum turnaround times from the time of sample collection to result return to the nurse being four days for smear microscopy at the peripheral lab, 10 days for flourochrome smear microscopy at the NTRL, two days for Xpert testing regardless of Xpert location, and 49 days for liquid culture results from the NTRL. Nurses were encouraged to inform patients of positive TB diagnoses the same day via phone, although, if the patient was unreachable by phone, the patient was informed at the next scheduled clinic appointment.
For the first 17 months of study conduct, all prospectively enrolled patients consented to 12 months of follow-up but this was shortened to 6 months of follow-up in December 2013 (17 months into study enrollment), in an attempt to reduce burden on study nurses.
Study procedures related to second key objective
Enrollment of the retrospective cohort (cohort R) was through chart abstraction of eligible patients who started ART in the 24 months before study initiation at one of the 22 study clinics. Chart abstraction procedures were similar to procedures described in previous studies . Data on important demographic and clinical characteristics were abstracted to maximize opportunities to explore and control for confounding, when estimating impact of the Xpert package on all-cause mortality. Since loss to follow-up (LTFU) from ART can account for as much as 75 % of all attrition (death plus LTFU) in ART programs, and because incidence of death following LTFU ranges from 20 % to 60 % and failure to adjust mortality estimates for death among LTFU patients could bias estimates of intervention effect [40, 46], tracing patients LTFU was considered essential to answer the second key question. Tracing for mortality ascertainment purposes, was conducted through two methods in all cohorts. Firstly, up to five telephone calls and two home visits were used to determine outcomes of patients LTFU; in the retrospective cohort this occurred following documentation that the patient was >90 days late for a scheduled appointment, whereas in the prospective cohort this tracing started the day following a missed appointment. Secondly, all patients that remained LTFU following telephonic and home visit tracing activities were searched for in Botswana’s national mortality database.
Since about 2,429 (28 %) of 8,565 patients eligible for the prospective cohort did not enroll due to logistical constraints (see above), a protocol amendment was approved in April 2014, that allowed retrospective chart abstraction of the missed prospective patients. By abstracting key baseline data (e.g., baseline CD4 count) and outcome data (e.g., vital status) on the 2,429 missed patients, investigators will be able to: (1) estimate if the prospective cohort is truly representative of new HIV clinic enrollees at the 22 study sites during study conduct, and (2) use appropriate methods to explore effect of non-response .
For the first key study question, we will employ a mixed-model approach similar to that presented by Hussey and Hughes (2007) . A generalized linear mixed model will be fit to the data. The dependent variable is dichotomous, indicating whether the diagnostic algorithm detected TB or not (only those with true TB detected by liquid culture will be included in the sensitivity analysis). A fixed effect for time will be included in the analysis to adjust for any time trends that might bias the pre-post comparison. A fixed effect for intervention condition (0 before Xpert implementation, 1 afterward) will also be included and is the test of the intervention effect. A random effect for clinic will be included to adjust for between-clinic variation. The intervention effect will be judged significant at p < 0.05 with a two-tailed test.
For the second key question, crude and multivariable Cox proportional hazards regression models, accounting for study design, will be fit to the data with a fixed effect specified for intervention status, and random effect for clinic . Since there are three levels to the intervention, with cohort R receiving standard of care, cohort A receiving the ICF intervention, and cohort B receiving the ICF intervention plus the Xpert diagnostic algorithm, intervention effect will be coded as a three-level variable to represent the three study phases. Although the protocol-specified primary question aims to compare 6-month mortality in cohort R versus cohort B, investigators will examine for any dose-response effect across cohorts R, A, and B, which could add data to inform interpretation of causal pathways . Importantly, the analysis will need to control for trends over time. Several reports from RLS, including Botswana , have reported improvements in baseline health status at ART initiation (e.g., higher median CD4 counts) and lower incidence of 6-month ART mortality over successive annual cohorts of ART enrollees. As described earlier, because cohorts R and B do not overlap during the stepped-wedge portion of this trial, a comparison of 6-month mortality rates in cohorts R and B cannot make use of the stepped-wedge design to control for secular trends . However, since most variation in 6-month ART mortality over successive annual ART cohorts is accounted for by changes in health status of ART enrollees (e.g., changes in baseline CD4 count), incorporation of these known risk factors for 6-month ART mortality into the multivariable model may fully account for secular mortality trends [7, 50].
In a secondary analysis, that excludes cohort R, we will compare 6-month ART mortality rates between cohorts A and B using analytic methods described by Moulton et al, fitting Cox proportional hazards models to the data with the underlying time frame being time since July 2012 (initiation date for the stepped-wedge component of the trial), fixed effect for intervention arm (Xpert device activation), and a random effect for clinic [26, 51]. We will also explore an alternate analytic approach, recommended by Hussey & Hughes, which utilizes a Poisson model, including fixed effect for intervention and time interval, and a random effect for cluster .
This research study was reviewed and approved by the CDC IRB C, the Health Research and Development Division of the Human Resource Development Council (HRDC) in Botswana, and the University of Pennsylvania IRB No.4. XPRES is registered at ClinicalTrials.gov (trial registration no. NCT02538952).
Prospective cohort enrollment started in July, 2012 and was completed by the end of March 2014. Retrospective cohort chart abstraction was complete by December 2015. Data entry is estimated to be complete by the end of September 2016. Data analysis for the primary study questions has not yet begun. Trial data will be reported according to published guidelines for cluster-randomised trials (Additional file 2).
The over-arching purpose of this project is to improve TB diagnostic and care services at 22 HIV care and treatment clinics through phased rollout of (1) strengthened ICF systems, and (2) 13 Xpert devices, while simultaneously answering important implementation science questions, concerning Xpert operationalization and impact.
The stepped-wedge study design was chosen for a number of reasons related to ethical, operational, and analytic needs, as described in the method’s section. During the course of study implementation, the operational advantages of the phased implementation approach have been particularly notable. In our RLS of Botswana, the phased implementation approach has allowed the limited human and financial resources to be focused on smaller, more manageable pieces of the whole project, one step at a time, rather than be spread thinly across study sites, as would be required in a parallel group CRT . Analytically, the stepped-wedge design allows multiple opportunities for controlling trends over time . Potential disadvantages, when compared with a parallel group CRT, include: (1) moderately lower ability to assign causality to the intervention, and (2) higher sample size requirements in most circumstances, because of unequal allocation ratios for most of the duration of stepped-wedge trials . The ethical, operational, and analytic advantages may help explain the increasing popularity of the stepped-wedge evaluation design, especially in RLS .
During trial conduct, several operational challenges were experienced, mainly related to lower than expected clinic enrolment rates, human resource constraints that reduced ability to enroll all study-eligible patients in the prospective cohort, and lower than expected prevalence of culture-positive TB at clinic enrollment. The declining HIV clinic enrolment rates probably reflect success of the HIV treatment program in reaching HIV-infected persons in prior years (i.e., during 2002–2011) , declining HIV incidence rates , and expanding numbers of alternate HIV clinics at which patients can receive care . The study team probably over-estimated the willingness of patients to wait at the clinic for their turn to enroll in the study. However, in response to the observation that 28 % of potentially study-eligible patients were not being enrolled in the prospective cohort, the study team wrote a protocol amendment that allowed retrospective chart abstraction for the missed prospective patients, which will allow investigators to quantify any potential selection bias incurred by non-response. The lower than expected prevalence of culture-positive TB at HIV clinic enrollment needs further investigation once all study data are available for analysis. Fueled by the HIV epidemic, TB case notification rates in Botswana increased from about 202/100,000 population in 1990 to about 600/100,000 in 1998, plateaued at this level during 1998 through 2007, and have since declined to about 470/100,000 in recent years . Increased ART coverage among HIV-infected persons might again explain declining national TB incidence and the lower-than-expected TB prevalence among HIV clinic enrollees in this study . In retrospect, the protocol-specified large sample sizes and resulting high pre-study power to answer the first two primary study questions, were important precautions in place to ensure any sample size shortfalls did not result in trial futility.
Although, several Xpert impact studies have been published after this trial started, the two key study questions have not yet been answered. Firstly, data validating the Botswana Xpert diagnostic algorithm have not yet been reported, and this is an important program evaluation activity . Secondly, among six trials that have compared all-cause mortality outcomes of study enrollees between microscopy and Xpert arms [17, 21, 54–57], none have observed Xpert impact on either morbidity or mortality outcomes, and only one was conducted exclusively among ART enrollees (Mupfumi et al) . Certain study limitations of the trial by Mupfumi et al, including small sample size (N = 424) and powering the study to detect differences in a composite outcome (death or TB) between study arms, mean that XPRES, with its larger sample size (N = 16,267) and powered to detect Xpert impact on 6-month mortality rates specifically, is still positioned to provide a valuable scientific contribution. In addition, the intervention in XPRES is different from interventions employed in previous Xpert impact trials [17, 21, 54–59]—it represents a package of strengthened ICF interventions, activation of the Xpert device, and improved tracing for patients late for ART clinic appointments. In real-world settings, ICF interventions are often implemented at a sub-optimal level of quality and consistency due to health system weakness , and strengthening health systems to improve ICF compliance is arguably as important as the rollout of a new TB diagnostic device . In addition, preventing treatment interruptions or LTFU during early ART through the tracing intervention, could contribute to reductions in all-cause, 6-month ART mortality rates [42, 43].
Acid fast bacilli
United States centers for disease control and prevention
Human resource development council
Institutional review board
Loss to follow-up
Ministry of health
National TB reference laboratory
Primary healthcare clinic
Persons living with HIV
Statistical analysis software
Tuberculosis; intensified tuberculosis case finding (ICF)
World Health Organization
Xpert MTB/RIF assay
Xpert package rollout evaluation using a stepped-wedge design
Gupta RK, Lucas SB, Fielding KL, Lawn SD. Prevalence of tuberculosis in post-mortem studies of HIV-infected adults and children in resource-limited settings: a systematic review and meta-analysis. AIDS. 2015;29(15):1987–2002.
Cox JA, Lukande RL, Lucas S, Nelson AM, Van Marck E, Colebunders R. Autopsy causes of death in HIV-positive individuals in sub-Saharan Africa and correlation with clinical diagnoses. AIDS Rev. 2010;12(4):183–94.
Lawn SD, Harries AD, Anglaret X, Myer L, Wood R. Early mortality among adults accessing antiretroviral treatment programmes in sub-Saharan Africa. AIDS. 2008;22(15):1897–908.
Wong EB, Omar T, Setlhako GJ, Osih R, Feldman C, Murdoch DM, et al. Causes of death on antiretroviral therapy: a post-mortem study from South Africa. PLoS One. 2012;7(10), e47542.
Centers for Disease C, Prevention. Differences Between HIV-Infected Men and Women in Antiretroviral Therapy Outcomes - Six African Countries, 2004–2012. MMWR Morb Mortal Wkly Rep. 2013;62(47):946–52.
Lawn SD, Wilkinson RJ, Lipman MC, Wood R. Immune reconstitution and “unmasking” of tuberculosis during antiretroviral therapy. Am J Respir Crit Care Med. 2008;177(7):680–5.
Auld AF, Ekra KA, Shiraishi RW, Tuho MZ, Kouakou JS, Mohamed F, et al. Temporal trends in treatment outcomes for HIV-1 and HIV-2-infected adults enrolled in Cote d’Ivoire’s national antiretroviral therapy program. PLoS One. 2014;9(5), e98183.
Auld AF, Tuho MZ, Ekra KA, Shiraishi RW, Mohamed F, Kouakou JS, et al. Temporal Trends in Mortality and Loss to Follow-up Among Children Enrolled in Cote d’Ivoire’s National Antiretroviral Therapy Program. Pediatr Infect Dis J. 2014;33(11):1134–40.
Auld AF, Mbofana F, Shiraishi RW, Alfredo C, Sanchez M, Ellerbrock TV, et al. Incidence and determinants of tuberculosis among adults initiating antiretroviral therapy—Mozambique, 2004–2008. PLoS One. 2013;8(1), e54665.
Chihota VN, Ginindza S, McCarthy K, Grant AD, Churchyard G, Fielding K. Missed Opportunities for TB Investigation in Primary Care Clinics in South Africa: Experience from the XTEND Trial. PLoS One. 2015;10(9), e0138149.
Parsons LM, Somoskovi A, Gutierrez C, Lee E, Paramasivan CN, Abimiku A, et al. Laboratory diagnosis of tuberculosis in resource-poor countries: challenges and opportunities. Clin Microbiol Rev. 2011;24(2):314–50.
Boehme CC, Nicol MP, Nabeta P, Michael JS, Gotuzzo E, Tahirli R, et al. Feasibility, diagnostic accuracy, and effectiveness of decentralised use of the Xpert MTB/RIF test for diagnosis of tuberculosis and multidrug resistance: a multicentre implementation study. Lancet. 2011;377(9776):1495–505.
Dheda K, Gumbo T, Gandhi NR, Murray M, Theron G, Udwadia Z, et al. Global control of tuberculosis: from extensively drug-resistant to untreatable tuberculosis. Lancet Respir Med. 2014;2(4):321–38.
Small PM, Pai M. Tuberculosis diagnosis—time for a game change. N Engl J Med. 2010;363(11):1070–1.
Steingart KR, Schiller I, Horne DJ, Pai M, Boehme CC, Dendukuri N. Xpert(R) MTB/RIF assay for pulmonary tuberculosis and rifampicin resistance in adults. Cochrane Database Syst Rev. 2014;1, CD009593.
Lawn SD, Meintjes G, McIlleron H, Harries AD, Wood R. Management of HIV-associated tuberculosis in resource-limited settings: a state-of-the-art review. BMC Med. 2013;11:253.
Churchyard GJ, Stevens WS, Mametja LD, McCarthy KM, Chihota V, Nicol MP, et al. Xpert MTB/RIF versus sputum microscopy as the initial diagnostic test for tuberculosis: a cluster-randomised trial embedded in South African roll-out of Xpert MTB/RIF. Lancet Glob Health. 2015;3(8):e450–7.
Lawn SD, Mwaba P, Bates M, Piatek A, Alexander H, Marais BJ, et al. Advances in tuberculosis diagnostics: the Xpert MTB/RIF assay and future prospects for a point-of-care test. Lancet Infect Dis. 2013;13(4):349–61.
Sanson-Fisher RW, D’Este CA, Carey ML, Noble N, Paul CL. Evaluation of systems-oriented public health interventions: alternative research designs. Annu Rev Public Health. 2014;35:9–27.
Lawn SD, Harries AD. Reducing tuberculosis-associated early mortality in antiretroviral treatment programmes in sub-Saharan Africa. AIDS. 2011;25(12):1554–5.
Mupfumi L, Makamure B, Chirehwa M, Sagonda T, Zinyowera S, Mason P, et al. Impact of Xpert MTB/RIF on Antiretroviral Therapy-Associated Tuberculosis and Mortality: A Pragmatic Randomized Controlled Trial. Open Forum Infect Dis. 2014;1(1):ofu038.
World Health Organization. Rapid Implementation of the Xpert MTB/RIF diagnostic test: 2011. Available at: http://whqlibdoc.who.int/publications/2011/9789241501569_eng.pdf. Accessed 18 Aug 2015.
Ministry of Health, Republic of Botswana. National Tuberculosis Programme Manual. Seventh Edition. 2011.
Mdege ND, Man MS, Taylor Nee Brown CA, Torgerson DJ. Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation. J Clin Epidemiol. 2011;64(9):936–48.
Maclure M. Explaining pragmatic trials to pragmatic policy-makers. CMAJ. 2009;180(10):1001–3.
Moulton LH, Golub JE, Durovni B, Cavalcante SC, Pacheco AG, Saraceni V, et al. Statistical design of THRio: a phased implementation clinic-randomized study of a tuberculosis preventive therapy intervention. Clin Trials. 2007;4(2):190–9.
Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28(2):182–91.
Hayes R, Moulton LH. Cluster Randomized Trials. Boca Raton: CRC Press; 2009.
Getahun H, Kittikraisak W, Heilig CM, Corbett EL, Ayles H, Cain KP, et al. Development of a standardized screening rule for tuberculosis in people living with HIV in resource-constrained settings: individual participant data meta-analysis of observational studies. PLoS Med. 2011;8(1), e1000391.
Lawn SD, Brooks SV, Kranzer K, Nicol MP, Whitelaw A, Vogt M, et al. Screening for HIV-Associated Tuberculosis and Rifampicin Resistance before Antiretroviral Therapy Using the Xpert MTB/RIF Assay: A Prospective Study. PLoS Med. 2011;8(7), e1001067.
Bassett IV, Wang B, Chetty S, Giddy J, Losina E, Mazibuko M, et al. Intensive tuberculosis screening for HIV-infected patients starting antiretroviral therapy in Durban, South Africa. Clin Infect Dis. 2010;51(7):823–9.
Lawn SD, Edwards DJ, Kranzer K, Vogt M, Bekker LG, Wood R. Urine lipoarabinomannan assay for tuberculosis screening before antiretroviral therapy diagnostic yield and association with immune reconstitution disease. AIDS. 2009;23(14):1875–80.
Lawn SD, Kranzer K, Edwards DJ, McNally M, Bekker LG, Wood R. Tuberculosis during the first year of antiretroviral therapy in a South African cohort using an intensive pretreatment screening strategy. AIDS. 2010;24(9):1323–8.
Shah S, Demissie M, Lambert L, Ahmed J, Leulseged S, Kebede T, et al. Intensified tuberculosis case finding among HIV-Infected persons from a voluntary counseling and testing center in Addis Ababa, Ethiopia. J Acquir Immune Defic Syndr. 2009;50(5):537–45.
Cain KP, McCarthy KD, Heilig CM, Monkongdee P, Tasaneeyapan T, Kanara N, et al. An algorithm for tuberculosis screening and diagnosis in people with HIV. N Engl J Med. 2010;362(8):707–16.
World Health Organization. African Health Observatory: Better information, better action for health. Available at: http://www.aho.afro.who.int/profiles_information/index.php/Botswana:Analytical_summary_-_Tuberculosis. Accessed 18 Aug 2015.
Ministry of Health, Ethiopia. Incidence of Tuberculosis in Ethiopia. Available at: http://www.tradingeconomics.com/ethiopia/incidence-of-tuberculosis-per-100-000-people-wb-data.html. Accessed 18 Aug 2015.
World Health Organization. Global Tuberculosis Report-2014. Available at: http://www.who.int/tb/publications/global_report/gtbr14_main_text.pdf. Accessed 18 Aug 2015.
Hayes RJ, Bennett S. Simple sample size calculation for cluster-randomized trials. Int J Epidemiol. 1999;28(2):319–26.
Bisson GP, Gaolathe T, Gross R, Rollins C, Bellamy S, Mogorosi M, et al. Overestimates of survival after HAART: implications for global scale-up efforts. PLoS One. 2008;3(3), e1725.
Braitstein P, Brinkhof MW, Dabis F, Schechter M, Boulle A, Miotti P, et al. Mortality of HIV-1-infected patients in the first year of antiretroviral therapy: comparison between low-income and high-income countries. Lancet. 2006;367(9513):817–24.
Brinkhof MW, Dabis F, Myer L, Bangsberg DR, Boulle A, Nash D, et al. Early loss of HIV-infected patients on potent antiretroviral therapy programmes in lower-income countries. Bull World Health Organ. 2008;86(7):559–67.
Cornell M, Lessells R, Fox MP, Garone DB, Giddy J, Fenner L, et al. Mortality among adults transferred and lost to follow-up from antiretroviral therapy programmes in South Africa: a multicenter cohort study. J Acquir Immune Defic Syndr. 2014;67(2):e67–75.
Ansari NA, Kombe AH, Kenyon TA, Hone NM, Tappero JW, Nyirenda ST, et al. Pathology and causes of death in a group of 128 predominantly HIV-positive patients in Botswana, 1997–1998. Int J Tuberc Lung Dis. 2002;6(1):55–63.
Auld AF, Mbofana F, Shiraishi RW, Sanchez M, Alfredo C, Nelson LJ, et al. Four-Year Treatment Outcomes of Adult Patients Enrolled in Mozambique’s Rapidly Expanding Antiretroviral Therapy Program. PLoS One. 2011;6(4), e18453.
Geng EH, Glidden DV, Bangsberg DR, Bwana MB, Musinguzi N, Nash D, et al. A causal framework for understanding the effect of losses to follow-up on epidemiologic analyses in clinic-based cohorts: the case of HIV-infected patients on antiretroviral therapy in Africa. Am J Epidemiol. 2012;175(10):1080–7.
Seaman SR, White IR. Review of inverse probability weighting for dealing with missing data. Stat Methods Med Res. 2013;22(3):278–95.
Lucas RM, McMichael AJ. Association or causation: evaluating links between “environment and disease”. Bull World Health Organ. 2005;83(10):792–5.
Farahani M, Vable A, Lebelonyane R, Seipone K, Anderson M, Avalos A, et al. Outcomes of the Botswana national HIV/AIDS treatment programme from 2002 to 2010: a longitudinal analysis. Lancet Glob Health. 2014;2(1):e44–50.
Auld AF, Shiraishi R, Couto A, Mbofana F, Xavier C, Alfredo C, et al. Evaluation of outcome trends and determinants and new models of service delivery among more than 300,000 adults starting antiretroviral therapy in Mozambique during 2004–2013. JAIDS. 2016;(in press).
Durovni B, Saraceni V, Moulton LH, Pacheco AG, Cavalcante SC, King BS, et al. Effect of improved tuberculosis screening and isoniazid preventive therapy on incidence of tuberculosis and death in patients with HIV in clinics in Rio de Janeiro, Brazil: a stepped wedge, cluster-randomised trial. Lancet Infect Dis. 2013;13(10):852–8.
UNAIDS. Global report: UNAIDS report on the global AIDS epidemic 2013. Available at: http://www.unaids.org/en/media/unaids/contentassets/documents/epidemiology/2013/gr2013/UNAIDS_Global_Report_2013_en.pdf. Accessed 14 Dec 2015.
Lawn SD, Harries AD, Williams BG, Chaisson RE, Losina E, De Cock KM, et al. Antiretroviral therapy and the control of HIV-associated tuberculosis. Will ART do it? Int J Tuberc Lung Dis. 2011;15(5):571–81.
Yoon C, Cattamanchi A, Davis JL, Worodria W, den Boon S, Kalema N, et al. Impact of Xpert MTB/RIF testing on tuberculosis management and outcomes in hospitalized patients in Uganda. PLoS One. 2012;7(11), e48599.
Theron G, Zijenah L, Chanda D, Clowes P, Rachow A, Lesosky M, et al. Feasibility, accuracy, and clinical effect of point-of-care Xpert MTB/RIF testing for tuberculosis in primary-care settings in Africa: a multicentre, randomised, controlled trial. Lancet. 2014;383(9915):424–35.
Cox HS, Mbhele S, Mohess N, Whitelaw A, Muller O, Zemanay W, et al. Impact of Xpert MTB/RIF for TB diagnosis in a primary care clinic with high TB and HIV prevalence in South Africa: a pragmatic randomised trial. PLoS Med. 2014;11(11), e1001760.
Calligaro GL, Theron G, Khalfey H, Peter J, Meldau R, Matinyenya B, et al. Burden of tuberculosis in intensive care units in Cape Town, South Africa, and assessment of the accuracy and effect on patient outcomes of the Xpert MTB/RIF test on tracheal aspirate samples for diagnosis of pulmonary tuberculosis: a prospective burden of disease study with a nested randomised controlled trial. Lancet Respir Med. 2015;3(8):621–30.
Durovni B, Saraceni V, van den Hof S, Trajman A, Cordeiro-Santos M, Cavalcante S, et al. Impact of replacing smear microscopy with Xpert MTB/RIF for diagnosing tuberculosis in Brazil: a stepped-wedge cluster-randomized trial. PLoS Med. 2014;11(12), e1001766.
van Kampen SC, Susanto NH, Simon S, Astiti SD, Chandra R, Burhan E, et al. Effects of Introducing Xpert MTB/RIF on Diagnosis and Treatment of Drug-Resistant Tuberculosis Patients in Indonesia: A Pre-Post Intervention Study. PLoS One. 2015;10(6), e0123536.
This research has been supported by the President’s Emergency Plan for AIDS Relief (PEPFAR) through the U.S. Centers for Disease Control and Prevention. We thank all study participants who made this research possible.
Availability of data and materials
The data supporting the content and discussion of the study protocol are included within the article and its additional files.
Authors AFA, SP, TA, AF, RB, HA, JB, JS, TVE, and AD conceived the study. Protocol writing was led by AFA and TA, with contributions from all authors. Design of protocol amendments were led by AFA and TA, with contributions from all authors. Power analysis and analytic approach were advised and implemented by AFA and SP. Authors TA, AF, RB, AM, JB, AFA, and SG, were primarily responsible for study implementation but all authors contributed. All authors contributed to study monitoring and supervision. All authors helped write and review the paper for intellectual content. AFA drafted the final version of the paper. All authors have approved the final manuscript submitted.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
Ethical approvals for this study were obtained from the U.S. Centers for Disease Control and Prevention (CDC) Institutional Review Board (IRB) C, the Health Research and Development Division of the Human Resource Development Council (HRDC) in Botswana, and the University of Pennsylvania IRB No.4. These are available in Additional file 2. Written informed consent was obtained from all prospectively enrolled patients. A waiver of informed consent was granted for abstraction of data from charts of retrospective cohort patients under 45CFR 46.116 (d).
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
Sources of support
This research has been supported by the President’s Emergency Plan for AIDS Relief (PEPFAR) through the U.S. Centers for Disease Control and Prevention.
About this article
Cite this article
Auld, A.F., Agizew, T., Pals, S. et al. Implementation of a pragmatic, stepped-wedge cluster randomized trial to evaluate impact of Botswana’s Xpert MTB/RIF diagnostic algorithm on TB diagnostic sensitivity and early antiretroviral therapy mortality. BMC Infect Dis 16, 606 (2016). https://doi.org/10.1186/s12879-016-1905-4
- Xpert MTB/RIF
- Diagnostic accuracy
- Antiretroviral therapy
- People living with HIV
- Stepped-wedge cluster randomized trial