Development of a clinical risk score for the prediction of Pneumocystis jirovecii pneumonia in hospitalised patients

Abstract

Background

The performance and availability of invasive and non-invasive investigations for the diagnosis of Pneumocystis jirovecii pneumonia (PCP) vary across clinical settings. Estimating the pre-test probability of PCP is essential to the optimal selection and interpretation of diagnostic tests, such as the 1,3-β-D-glucan assay (BDG), for the prioritization of bronchoscopy, and to guide empiric treatment decisions. We aimed to develop a multivariable risk score to estimate the pre-test probability of PCP.

Methods

The score was developed from a cohort of 626 individuals who underwent bronchoscopy for the purposes of identifying PCP in a Canadian tertiary-care centre, between 2015 and 2018. We conducted a nested case-control study of 57 cases and 228 unmatched controls. Demographic, clinical, laboratory, and radiological data were included in a multivariable logistic regression model to estimate adjusted odds ratios for PCP diagnosis. A clinical risk score was derived from the multivariable model and discrimination was assessed by estimating the score’s receiver operating characteristic curve.

Results

Participants had a median age of 60 years (interquartile range [IQR] 49–68) and 115 (40%) were female; 40 (14%) had HIV and 49 (17%) had a solid organ transplant (SOT). The risk score included prior SOT or HIV with CD4 ≤ 200/µL (+ 2), serum lactate dehydrogenase ≥ 265.5 IU/mL (+ 2), radiological pattern typical of PCP on chest x-ray (+ 2) or CT scan (+ 2.5), and PCP prophylaxis with trimethoprim-sulfamethoxazole (-3) or other antimicrobials (-2). The median score was 4 points (IQR, 2-4.5) corresponding to a 28% probability of PCP. The risk prediction model had good discrimination with a c-statistic of 0.79 (0.71–0.84). Given the operating characteristics of the BDG assay, scores ≤ 3 in patients without HIV, and ≤ 5.5 in those with HIV, paired with a negative BDG, would be expected to rule out PCP with 95% certainty.

Conclusion

We propose the PCP Score to estimate pre-test probability of PCP. Once validated, it should help clinicians determine which patients to refer for invasive investigations, when to rely on serological testing, and in whom to consider pre-emptive treatment.

Introduction

Pneumocystis jirovecii pneumonia (PCP) is an opportunistic fungal infection that can result in life-threatening respiratory failure, most often in the context of immune compromise [1]. With greater access to antiretroviral therapy (ART), there has been a decline in the incidence of PCP among people with human immunodeficiency virus (HIV) in many countries. However, inequalities in access to ART persist both between and within countries [2], and non-HIV infected populations at risk of PCP are expanding due to advances in biologic immunotherapies, chemotherapies, and transplantation, along with a rising incidence of several hematologic malignancies [3, 4]. In most jurisdictions, tracking and reporting PCP is not mandatory; nonetheless, recent studies have reported a rising incidence of PCP in several countries [5, 6], and have estimated over 400,000 annual cases of PCP worldwide [7, 8]. Notwithstanding forthcoming advancements in PCP treatment [9], ensuring timely and accurate diagnosis of the infection is critical to improving outcomes.

Definite diagnosis of PCP relies on the detection of the organism by cytological staining, immunofluorescence, or quantitative polymerase chain reaction (qPCR) in samples obtained with invasive testing such as bronchoalveolar lavage (BAL) or transbronchial biopsy. Non-invasive testing, including sputum induction and nasopharyngeal aspiration, can be used to detect PCP with high sensitivity and/or specificity in several clinical settings [10]. Serological investigations, such as 1,3-β-D-glucan (BDG), can also be useful to inform diagnosis in the right clinical context [11]. The performance of each of these tests may differ between individuals with and without HIV: respiratory samples obtained from patients with HIV and PCP contain large amounts of P. jirovecii and few recruited lymphocytes and neutrophils, whereas samples from patients without HIV are more likely to contain numerous inflammatory cells, including neutrophils, yet relatively fewer P. jirovecii organisms [12,13,14]. Furthermore, because PCP is ubiquitous in the environment and healthy individuals can become asymptomatic carriers, certain test results can be challenging to interpret; for example, detection of PCP on qPCR can reflect colonisation rather than infection [10].

In any given clinical scenario, the pre-test probability of PCP depends on a number of factors, such as host susceptibility, clinical characteristics, and radiological findings [15]. Estimating the pre-test probability of PCP becomes essential for the optimal selection and interpretation of PCP diagnostic tests and for the prioritisation of bronchoscopy. Patients who are at high probability of disease may require empiric therapy and early bronchoscopy. Patients at lower probability of disease might benefit from non-invasive tests (including BDG) and avoid the potential harms of unnecessary pre-emptive treatment (e.g., adverse drug events or delay in obtaining an accurate alternative diagnosis). Our aim was to derive a multivariable risk prediction score to estimate the pre-test probability of PCP to facilitate the selection and interpretation of tests and to guide pre-emptive treatment decisions.

Methods

The McGill University Health Centre (Montreal, Canada) is a 770-bed tertiary care hospital and referral center for solid organ transplantation (kidney, liver, pancreas, and heart), autologous and allogeneic stem cell transplantation, oncology, HIV, and connective tissue disease. Cases of suspected PCP are referred to internal medicine, infectious diseases, and respiratory medicine services for diagnosis and management. On average, our center cares for 50 suspected cases per year.

Table 1 Characteristics of PCP cases and controls included in study population and PCP score derivation

To develop a clinical risk prediction score, we screened all consecutive BALs performed at our centre between January 2015 and January 2018 wherein calcofluor was used to evaluate for the presence of Pneumocystis. Individuals for whom the documented indication for bronchoscopy was not suspected PCP were subsequently excluded. From this cohort, cases were defined as individuals with a cytologically-confirmed diagnosis of PCP. For each case, four unmatched controls who underwent bronchoscopy during the same period and for the same indication, but who tested negative for PCP, were randomly selected. For each case and control, we manually extracted information about demographics, co-morbidities, vital signs on presentation, medications prior to admission including immunosuppressive agents and PCP prophylaxis, course in hospital, laboratory testing, and thoracic imaging as interpreted by a chest radiologist (Table 1).

Comorbidities and blood tests

Comorbidities considered in our analyses include obstructive lung disease, diabetes mellitus, HIV with a CD4 cell count equal to or less than 200/µL, solid organ transplantation, hematologic malignancy, stem cell transplantation, and connective tissue disease. Serum lactate dehydrogenase (LDH) was dichotomised using the Youden method to determine the optimal cut point above which LDH might be considered elevated in PCP (265.5 units/L in our population) [16]. Missing data for LDH (n = 34, including 4 cases) were substituted by the median level. BDG was not included in the multivariate model, as measurements were available only for 23 individuals (8%), reflecting changes in availability of the test during the study period.

Radiology

Radiographic findings from chest x-rays and thoracic CT scans were collected and categorised, and have been previously described [15]. Briefly, prior analyses from our group have shown that interstitial markings on chest x-ray or CT scan, as well as ground glass opacities on CT scan, are highly associated with diagnosis of PCP, whereas other radiographic features traditionally deemed suggestive of PCP (namely, septal thickening, crazy paving, and cystic changes) were not associated with the diagnosis [15]. Thus, for the present study, chest x-rays were dichotomised as typical for PCP if reported as demonstrating increased interstitial markings, and thoracic CT scans were identified as typical for PCP only if reported as demonstrating either increased interstitial markings or ground-glass opacities. A number of participants did not have a chest x-ray performed [21 participants (including 2 cases)], and others did not have CT scan performed [49 participants (including 6 cases)], but no participants were missing data for both imaging modalities. Missing values were coded as radiographic findings that were not typical for PCP for the missing modality.

Medications

PCP prophylaxis prior to hospital admission was categorised according to the drug prescribed: trimethoprim-sulfamethoxazole (TMP-SMX) at prophylactic dosing or greater (1 single strength tablet 3 times weekly or more), or alternative agents (including dapsone and atovaquone). Participants were deemed to be on non-physiological doses of systemic steroids prior to hospitalisation if receiving ≥ 15 mg/day of prednisone, or equivalent, for at least 3 weeks. A variable combining other non-steroidal immunosuppressive medications, including mycophenolate, cyclosporin, tacrolimus, tumor necrosis factor inhibitors, rituximab, and methotrexate was created.

Statistical analyses

Data were presented as counts and percentages for categorical variables. The characteristics of cases and controls were compared using Pearson’s Chi-square testing. We used logistic regression to evaluate for univariate associations between PCP and relevant risk factors selected a priori based on previous studies. Those factors assessed included: age; biological sex; HIV with CD4 cell count ≤ 200/µL or solid organ transplantation; hematologic malignancy; stem cell transplantation; rheumatologic disease including scleroderma, rheumatoid arthritis, systemic lupus erythematosus, and others; exogenous steroid therapy; dichotomised serum LDH level; radiographic imaging findings as described above; and receipt of PCP prophylaxis [1, 3, 15]. Age and sex were chosen a priori to remain in the final model. Otherwise, variables for which p < 0.2 were considered for inclusion in the multivariable analysis. Odds ratios (ORs) and 95% confidence intervals (95% CIs) were estimated using multivariable logistic regression with stepwise forward variable selection, using the Firth method to reduce bias given the limited number of events per variable [17]. Goodness-of-fit was evaluated with likelihood ratio testing (LRT), whereby variables for which a statistically significant improvement in fit was identified with LRT (p < 0.05) were included in the multivariate model. Multicollinearity was deemed significant if the variance inflation factor exceeded 5 [18]. Internal validation of the final model was performed by bootstrapping with 1000 repetitions to generate a corrected c-statistic with confidence interval [19]. All reported P values were two-tailed, and statistical significance was defined as p < 0.05.

A clinical risk score was derived from the final multivariable regression model using the methodology proposed by Sullivan et al. [20] In brief, each risk factor was assigned a reference value, which was attributed 0 points in the scoring system. From the regression coefficients in the multivariate model, a regression unit was selected, and the coefficient for each variable was redefined in terms of regression units. The number of regression units corresponding to 1 point in the risk score was selected, and consequently the number of points associated with each category of each predictor variable was calculated. The theoretical range of point totals was determined, and the risks associated with each point total were derived from the regression model. To verify the internal coherence of the risk score, “regression” scores were computed for each participant as the sum of dependent predictors variables weighted by relevant regression coefficients, and “clinical” scores were calculated using discrete risk score point totals for each predictor. The correlation between regression and clinical scores was evaluated. Total scores were obtained by adding points attributed to each predictive variable, with higher scores corresponding to a higher probability of PCP. The point cut-off that maximised both sensitivity and specificity was established using nonparametric estimation of the score’s receiver operating characteristic curve. Reporting followed TRIPOD guidelines for prediction model development. Finally, we contextualised the risk score to establish a pretest probability threshold at which the BDG assay would be predicted to be most informative [11], analogous to the use of the D-dimer with pretest probability of venous thromboembolism [21]. All analyses were performed using STATA version 17.0 (StataCorp LP, USA). The McGill University Health Centre Institutional Ethics Review Board approved the study.

Results

Of 860 unique patient samples tested for PCP, 626 (72.8%) had suspected PCP documented as the primary indication for referral and were thus deemed eligible for inclusion. From this group, 57 unique cases (9.1% positivity) were identified, and 228 unmatched controls (4:1) without PCP were randomly selected.

Clinical characteristics of cases and controls are presented in Table 1. Cases and controls were similar with respect to age, sex, proportion with a chronic obstructive lung disease, diabetes, hematologic malignancy, solid organ cancer, or immunomodulatory medications (including steroids). Cases were more likely than controls to have a history of HIV or solid organ transplantation, an elevated serum LDH above the established cut-off, and radiographic findings typical for PCP, whereas controls were more likely to have CT findings atypical for PCP (pleural effusion, nodular pattern), and to be receiving PCP prophylaxis.

Table 2 Odds ratios and 95% confidence intervals for univariate and multivariate regression models

In univariate analyses, age > 60 years, HIV with CD4 cell count ≤ 200/µL or solid organ transplantation, hematologic malignancy, stem cell transplantation, connective tissue disease, exogenous corticosteroids, serum LDH, PCP prophylaxis, CXR findings typical of PCP, CT findings typical of PCP, and CT findings atypical of PCP met criteria for consideration in multivariable model building. (Table 2)

In the multivariate analysis, a history of either HIV with CD4 cell count ≤ 200/µL or a prior solid organ transplantation (OR 3.16; 95% CI 1.48–6.77), receipt of PCP prophylaxis (OR 0.15, 95% CI 0.03–0.83 for trimethoprim-sulfamethoxazole; OR 0.30, 95% CI 0.07–1.42 for alternative agents), serum LDH greater or equal to 265.5IU/mL (OR 3.02; 95% CI 1.43–6.36), chest x-ray findings typical of PCP (OR 2.89, 95% CI 1.43–5.82), and CT findings typical of PCP (OR 4.44, 95% CI 1.96–10.1) remained independently associated with the diagnosis of PCP. After bootstrapping, the corrected c-statistic was found to be 0.79 (95% CI 0.73–0.86) (good discrimination).

Table 3 Variables independently associated with pneumocystis jirovecii pneumonia in multivariate logistic regression, and point attribution in PCP score

Based on these variables, the PCP (Pneumocystispneumonia) Score is presented in Table 3. The correlation between “regression” risk scores, computed directly from the regression coefficients, and “clinical” risk scores, weighted for ease of use in clinical settings, was high (r = 0.999). The predicted probability of PCP as a function of the risk score is reported in Fig. 1, with possible point totals ranging from − 3 points (0.6% probability of PCP) to 8.5 points (85.4% probability of PCP). The median risk score was 4 points (interquartile range [IQR] 2 to 4.5 points) corresponding to a probability of PCP of 28.1%. The risk prediction model had good calibration and discrimination, with a mean area under the receiver operating characteristic curve of 0.77 (95% CI, 0.71–0.84). (Fig. 2). The risk score threshold which optimised both sensitivity and specificity was 4 points, corresponding to a sensitivity of 77% and a specificity of 64% for diagnosis of PCP. Considering the operating characteristics of the BDG assay [11], a cut-off of ≤ 3 points in patients without HIV and ≤ 5.5 points in patients with HIV, when paired with a negative BDG, would be expected to have a post-test probability of < 5%. More stringent cut-points could be chosen if a smaller post-test probability with the BDG assay were desired. A score less than or equal to -1 points was associated with a less than 2% probability of PCP. (Fig. 1)

Fig. 1

Probability of Pneumocystis jirovecii pneumonia (PCP) and PCP score point total

Fig. 2

Discrimination of the PCP score, assessed by the area under the receiver operating characteristic curve. AUC, area under the curve

The bottom-most fine dotted line denotes the score of -1 points, associated with a less than 2% probability of PCP. Considering the operating characteristics of the BDG assay [11], in HIV-negative individuals with a PCP score of ≤ 3 points, corresponding to a pre-test probability of < 20%, a negative BDG assay would be expected to rule out PCP with 95% certainty (middle line, short dashes). In HIV-positive individuals with a PCP score of ≤ 5.5 points, corresponding to a pre-test probability of < 50%, a negative BDG assay would be expected to rule out PCP with 95% certainty (top-most line, long dashes).

Discussion

We developed the PCP Score, a clinical prediction score to better estimate pre-test probability of PCP, in a population of hospitalised patients in whom the infection was suspected. If validated, the PCP Score could be used at the bedside to guide the choice of PCP testing, aid in prioritisation of bronchoscopy, and/or inform decisions about empiric therapy. Our risk score model had good calibration and discrimination (c-statistic 0.77), as did the multivariable logistic regression model on which it was based (c-statistic 0.79).

Few studies have proposed clinical risk scores for PCP, and none have done so in a general population of patients both with and without HIV, or outside of the critical care setting. Maartens et al. derived two clinical prediction scores for PCP in HIV-infected individuals using variables that are available in resource-limited settings, yet these scores are of limited utility in a general hospitalised patient population [22]. Azoulay et al. developed a multivariable prediction model for PCP in patients with hematologic malignancies admitted to the ICU in a cohort of 1099 individuals (134 with PCP). In their model, lymphoproliferative disease, lack of PCP prophylaxis, non-alveolar pattern on chest X-ray, and more than 3 days between onset of respiratory symptoms and ICU admission were associated with a high risk of PCP, and age > 50 years, shock, and pleural effusions were associated with a lower PCP risk [23]. However, chest radiographs are not the best imaging modality for PCP, particularly in immunocompromised patients who may not develop classic, overt radiographic findings [15], and time between respiratory symptom onset and ICU admission might be confounded by ICU bed capacity and institutional admission policies. Unfortunately, this risk score failed to predict PCP risk in an external validation study among 141 patients in South Korea [24]. Like Azoulay’s, our model found typical thoracic imaging and lack of PCP prophylaxis to be most predictive of PCP, while also identifying other variables to help estimate pre-test probability of infection.

The main strengths of our study are the inclusion of a mixed cohort of patients with and without HIV and the use of a control group comprised of any hospitalised patient in whom PCP was the indication for bronchoscopy. Our study population better represents a general patient population at risk of PCP and is more broadly applicable than prior risk prediction scores that have been restricted to subgroups. Of note, our population did not include patients whose respiratory status was too tenuous to safely undergo bronchoscopy, or who solely underwent sputum analyses; while inclusion of such patients might have further improved generalisability of the PCP Score, the rate of false positives might have increased. Cases in our study had cytologically proven PCP on BAL samples from the lower respiratory tract, minimising concerns related to overdiagnosis, or misdiagnosis of, for example, PCP colonisation detected by PCR from upper respiratory tract samples.

Our study has several limitations. This was a single institution study at a tertiary care referral centre for oncology and transplantation; therefore, there is substantial experience with PCP in our institution and suspected cases are usually reviewed by physicians who specialise in care of immunocompromised patients. This includes dedicated chest radiologists who interpret the imaging, and who, like in real-world clinical practice, were not blinded to the context in which the imaging was ordered. Some classically described risk factors for PCP, including corticosteroid therapy, hematologic malignancy, and stem cell transplantation, were not independently associated with PCP in our population. This may be attributable to use of prophylactic antimicrobial therapy. Alternatively, it may reflect the characteristics of our study population itself, comprised of individuals in whom there was clinical suspicion of PCP sufficient to request a bronchoscopy; consequently, some classical risk factors for PCP might have been distributed equally between cases and controls. LDH measurements were missing for 34 participants, which is a limitation of our study. However, values were missing at random, and missing LDH data were substituted with the median LDH value, which, if anything, would only have attenuated the strength of the association.

Another limitation is that BDG was available for only 8% of our study population, and thus was not included in our analyses. Nonetheless, we suggest that the PCP score can help refine the selection of patients in whom BDG will meaningfully alter the post-test probability of PCP. For example, based on our published analyses of the characteristics of the BDG assay, a PCP score ≤ 3 points in individuals without HIV, or a PCP score of ≤ 5.5 points in those with HIV, paired with a negative BDG test, would be expected to rule out PCP with 95% certainty [11]. Although prevalent worldwide, PCP can be relatively uncommon in a single institution. The number of cases in our analysis matched or exceeded that in most comparable studies, nonetheless, to ensure adequate power, we combined certain risk factors, such as history of HIV with CD4 ≤ 200/µL and SOT, which has advantages and disadvantages. Our score is useful in a general population of hospitalised patients at risk of PCP, yet it could be refined by increasing the number of cases in relevant subgroups and examining all comorbidities as separate risk factors. Finally, our population did not include enough cases for a distinct validation cohort: our findings require external validation in a larger, multi-centre population, in non-tertiary care institutions, and in low- and middle-income countries.

Despite these limitations, we hope that the PCP Score will be validated and will help clinicians refine their assessment of the pre-test probability of PCP by categorising patients more reliably as low- or high risk, to help guide testing and management decisions.

Conclusion

We present the PCP Score, a multivariable risk prediction score for PCP in hospitalised patients in whom there is a clinical suspicion of PCP. At lower scores, corresponding to lower pre-test probabilities of PCP, non-invasive testing, such as the BDG assay, could appropriately be used to exclude PCP. At higher scores, corresponding to higher pre-test probabilities of PCP, BDG will not be useful, and patients should be prioritised for invasive detection testing and pre-emptive antimicrobial therapy. If validated, the PCP score could therefore serve as a diagnostic stewardship tool. International collaborations with more patient data are needed to validate this score and render it generalisable to diverse and distinct healthcare settings and populations.