Study setting and participants
We conducted a secondary analysis of a data from a large prospective cohort study [7], which was designed to improve the evidence base for the WHO algorithm for the diagnosis of tuberculosis in seriously ill HIV-infected participants with current cough [3]. Recruitment for the main study took place at two secondary level hospitals in Cape Town, South Africa, serving communities with high HIV and tuberculosis prevalence: G.F. Jooste District Hospital from November 2011 until the hospital’s closure in February 2013, and Khayelitsha District Hospital from March 2013 until October 2014.
Inclusion criteria for the main study were: admission into the enrollment facility within the previous 24 h, ≥18 years of age, known HIV infection, cough of any duration, and at least one WHO-defined danger sign (respiratory rate > 30/min, fever > 39o C, pulse rate > 120/min, and unable to walk unaided). Exclusion criteria were: anti-tuberculosis treatment that was current, completed in the previous month, or defaulted in the past 6 months (isoniazid preventive therapy was allowed); exacerbation of either congestive cardiac failure or chronic obstructive pulmonary disease; and failure to provide a spontaneous or induced sputum specimen.
For the current study we added the inclusion criterion of a CRP and procalcitonin result, (funding for these two assays only became available after the start of the main study). We also only included participants fulfilling our a priori case definitions for tuberculosis, CAP, and/or PJP.
Case definitions
Tuberculosis: positive Mycobacterium tuberculosis culture from any site plus at least one symptom consistent with tuberculosis (cough, fever, night sweats, weight loss). CAP: cough ≤14 days plus one or more additional respiratory symptoms (sputum, breathlessness, chest pain, haemoptysis or fever) plus radiological evidence of pulmonary consolidation (confirmed by a radiologist) [14]. PJP: cough ≤ 3 months plus radiological evidence of diffuse bilateral interstitial infiltrates (confirmed by a radiologist) plus oxygen saturation ≤ 92% (adapted from Centers for Disease Control and Prevention case definition) [15].
Investigations
Three sputum specimens were obtained from each participant. One sample was sent for Gram stain, culture, and sensitivity, and two samples for smear examination with auramine staining for acid-fast bacilli (AFB) and liquid mycobacterial culture (BACTEC™ MGIT™ 960; Becton, Dickinson and Company, New Jersey, USA). Mycobaterial blood culture was done on all participants. Mycobacterial culture was done on other specimens when appropriate (e.g. pleural fluid).
CRP (Siemens Advia 1800), procalcitonin (Siemens Advia Centaur XP), and β-D-glucan assay (Fungitell™; Associates of Cape Cod, Inc., east Falmouth, MA, USA) were done on stored serum in a batch after the study, therefore these tests had no role in patient management. Laboratory staff were blinded to participant diagnosis and outcome. Assay range for CRP was 4-[304–336] mg/L, normal range was below 10 mg/L. Assay range for procalcitonin was < 0.02–75 μg/L, normal range below 0.02 μg/L.
Chest radiographs were performed on admission and retrospectively reviewed by a senior radiologist blinded to diagnoses and results of laboratory investigations.
Statistical analyses
All analyses were performed using Stata software version 13.0 (StataCorp Inc., College Station, Texas, USA).
Based on our fixed sample size of 210 participants with single respiratory infections, we explored precision to detect 90% sensitivity for each biomarker for the three target infections, aiming for a maximum ±10% variation in 95% confidence intervals (CIs). We estimated a range of CIs of binomial proportions using the Wilson-score interval for smaller sample sizes [16]. Since our data was not normally distributed, a second calculation was made using 85% of the original sample sizes as suggested by Lehmann et al. [17]. We estimated the 95% CIs of 90% sensitivity to be 83–94% for tuberculosis, 79–96% for CAP, and 69–99% for PJP. The small sample size for PJP accounted for wide 95% confidence intervals. Further details of these sample size calculations are provided in an additional table (see Additional file 1: Table S1).
To detect differences in CRP concentrations between the three target infections, we estimated power for a two-sample means test (assuming unequal variances), based on relevant literature. (Expected means for CRP were approximate due to lack of reported standard deviations for tuberculosis or CAP). Our study had 80% power and alpha of 0.05 (using 85% of the original sample size to account for non-normal distribution of CRP and procalcitonin concentrations) to detect a minimum mean concentration difference in CRP between tuberculosis and PJP of 36%, between CAP and PJP of 14%, and between CAP and tuberculosis of 14%, and a minimum mean concentration difference in procalcitonin of 50% between tuberculosis and PJP, 62% between CAP and PJP, and 62% between CAP and tuberculosis. We were unable to find data on sensitivity estimates for procalcitonin in all three target infections in HIV-infected individuals, therefore calculations were based exclusively on studies reporting CRP measures of diagnostic accuracy. Further details of these sample size calculations are provided in an additional table (see Additional file 1:, Table S2).
Diagnostic accuracy analyses for CRP and procalcitonin were performed for participants fulfilling criteria for one of the three single infection definitions. Participants with mixed infection were analysed separately. In clinical practice, differential diagnostic challenges usually present between two of the target infections and less commonly between all three, hence we calculated diagnostic accuracy measures between infection pairs in addition to each target infection versus the other two.
As distributions of both CRP and procalcitonin were not normally distributed, we used non-parametric statistical tests for continuous variables. Univariate associations between participant baseline characteristics in infection pairs were analysed using the Wilcoxon-Mann-Whitney test for continuous data, and Chi-square test (or Fisher’s Exact test if values in a cell were < 5), for categorical data. All probability tests were two-tailed. CRP and procalcitonin values below the detectable limit (BDL) of the assay were substituted with half BDL (in preference to substitution with the assay limit or with zero, both of which have been shown to bias parameter estimates) [18].
Receiver Operating Characteristic (ROC) area under the curve (AUC) analyses were used to explore potential cut-offs for CRP for each target infection using Liu’s index [19], which we then used to calculate diagnostic accuracy estimates. Cut-offs were also explored using the WHO 90% sensitivity recommendation for screening tests for tuberculosis [20]. To mitigate overfitting and improve accuracy of model prediction, we performed cross-validation on all ROC AUC’s exceeding 60% using k-fold cross-validation, as the dataset was too small for generation of a training set.
Since there are few studies on the diagnostic accuracy of procalcitonin for infections in HIV-infected patients, we explored cut-offs established for both lower respiratory tract infections (LRTI) and sepsis. Procalcitonin categories for LRTI were: < 0.1 μg/L, bacterial infection very unlikely; 0.1–0.25 μg/L, localised bacterial infection unlikely; 0.25–0.5 μg/L, localised bacterial infection possible; > 0.5 μg/L, suggestive of bacterial infection. Procalcitonin categories for systemic bacterial infection / sepsis were: 0.5–2 μg/L, systemic infection possible; 2–10 μg/L, suggestive of systemic infection; > 10 μg/L, severe systemic infection / septic shock [9].
This study conforms to the Standards for Reporting Diagnostic Accuracy Studies (STARD) guidelines [21].