A risk score for identifying methicillin-resistant Staphylococcus aureus in patients presenting to the hospital with pneumonia
© Shorr et al.; licensee BioMed Central Ltd. 2013
Received: 14 August 2012
Accepted: 31 May 2013
Published: 6 June 2013
Methicillin-resistant Staphylococcus aureus (MRSA) represents an important pathogen in healthcare-associated pneumonia (HCAP). The concept of HCAP, though, may not perform well as a screening test for MRSA and can lead to overuse of antibiotics. We developed a risk score to identify patients presenting to the hospital with pneumonia unlikely to have MRSA.
We identified patients admitted with pneumonia (Apr 2005 – Mar 2009) at 62 hospitals in the US. We only included patients with lab evidence of bacterial infection (e.g., positive respiratory secretions, blood, or pleural cultures or urinary antigen testing). We determined variables independently associated with the presence of MRSA based on logistic regression (two-thirds of cohort) and developed a risk prediction model based on these factors. We validated the model in the remaining population.
The cohort included 5975 patients and MRSA was identified in 14%. The final risk score consisted of eight variables and a potential total score of 10. Points were assigned as follows: two for recent hospitalization or ICU admission; one each for age < 30 or > 79 years, prior IV antibiotic exposure, dementia, cerebrovascular disease, female with diabetes, or recent exposure to a nursing home/long term acute care facility/skilled nursing facility. This study shows how the prevalence of MRSA rose with increasing score after stratifying the scores into Low (0 to 1 points), Medium (2 to 5 points) and High (6 or more points) risk. When the score was 0 or 1, the prevalence of MRSA was < 10% while the prevalence of MRSA climbed to > 30% when the score was 6 or greater.
MRSA represents a cause of pneumonia presenting to the hospital. This simple risk score identifies patients at low risk for MRSA and in whom anti-MRSA therapy might be withheld.
Methicillin-resistant Staphylococcus aureus (MRSA) is an important pathogen in a number of conditions ranging from pneumonia to bacteraemia . In critically ill patients, MRSA accounts for more than 70% of the S. aureus isolated [1, 2]. Because of the multitude of infections associated with MRSA along with its potential severity, this pathogen results in substantial morbidity and mortality .
MRSA has often been implicated as a cause of either hospital-acquired or ventilator-associated pneumonia (HAP or VAP) . In contrast, Streptococcus pneumoniae, Haemophilus influenzae, and Legionella spp. historically represent the most common bacterial causes of community-acquired pneumonia (CAP) . However, a number of bacteria historically thought confined to nosocomial processes are increasingly isolated in community-onset infections. This trend fostered the creation of the concept of healthcare-associated pneumonia (HCAP) . Although both patients with HCAP and CAP suffer the onset of their infections outside the hospital, those with HCAP are distinct because of their ongoing exposure to the healthcare system.
Unfortunately, reliance on the notion of HCAP to guide the use of broader spectrum antibiotics may result in overutilization of such agents. Moreover, HCAP as currently defined, appears to lack specificity as a screening test for resistant pathogens [5–7]. Prior studies evaluating risk factors for resistant pathogens in pneumonia presenting to the hospital suggest that grouping Gram-negative organisms such as Pseudomonas aeruginosa together with MRSA may be inappropriate [5–7]. These distinct pathogens possess different risk factors, or select risk factors for resistance may carry differential significance for the varying organisms. For example, theoretically, nursing home residence may be associated more with MRSA than P. aeruginosa. Additionally, clinicians are able to either add or withhold anti-MRSA therapy without compromising the extent of the Gram-negative coverage. The decision to prescribe an anti-MRSA agent can be decoupled from the option of Gram-negative treatment. Several agents exist which possess only Gram-positive activity and are essentially employed only for their anti-MRSA properties. Thus, it seems prudent to better recognize the specific variables that are associated with an increased risk for MRSA. Understanding these patient characteristics would more precisely allow clinicians to segregate persons at risk for MRSA from those unlikely to be infected with MRSA. Furthermore, identifying those at low risk for MRSA would facilitate antibiotic stewardship by rationally limiting unnecessary anti-MRSA therapy which would help contain costs and reduce future antibiotic resistance . Therefore, we conducted a retrospective analysis in order to develop a score to stratify patients presenting to the hospital with pneumonia by their risk for MRSA infection.
Study overview and patients
We conducted a multi-centre, retrospective analysis of patients admitted to the hospital with pneumonia between April 2005 and March 2009. The data are from 62 hospitals in the United States (US) via the Cerner Health Facts database (see Additional file 1). We included adults with a primary discharge diagnosis of pneumonia or a primary diagnosis of sepsis with a secondary diagnosis of pneumonia and no other source of infection by International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) codes (see Additional file 2). The codes were selected based on existing published studies [10–12] in combination with clinical judgment. We excluded those younger than 18 years of age and individuals with only nosocomial pneumonia (e.g., HAP, VAP). Nosocomial pneumonia was defined as occurring within 48 hours of hospitalization in accordance with the ATS/IDSA definitions . We only included persons with pneumonia and laboratory confirmation of a bacterial aetiology. Presence of a bacterial aetiology was determined based on a review of electronic health records for culture results (blood, respiratory secretions, pleural fluid) and urinary antigens for either S. pneumoniae or Legionella spp. We only reviewed cultures obtained within 48 hours of admission. The database contains only de-identified data and is HIPAA compliant. Because of this and the nature of the present analysis (review of existing records), this study was classified as exempt from Institutional Review Board (IRB) review.
Endpoints and covariates
The isolation of MRSA served as our primary endpoint. We compared those with MRSA to those infected with other bacterial organisms. Patients with MRSA isolated along with another pathogen were counted as MRSA for the purposes of this study. Polymicrobial status (more than one isolate) was captured as a separate variable in the study. The determination of methicillin-resistance was made at each individual site in accordance with local laboratory practices. We collected information regarding patient demographics, co morbidities (based on ICD-9-CM codes), severity of illness, and source of admission. Need for intensive care unit (ICU) admission represented our primary measure of disease severity. We identified patients (based on ICD-9-CM codes) with coronary artery disease (CAD), congestive heart failure (CHF), diabetes mellitus (DM), chronic obstructive pulmonary disease (COPD), stroke, dementia, or chronic kidney disease necessitating haemodialysis (HD). We calculated Charlson Co morbidity scores to determine the extent of co morbidities . We additionally categorized patients as to whether they met criteria for HCAP . We defined HCAP as present if any one of the following criteria were met: nursing home, long term acute care facility, or skilled nursing facility (NH/LTC/SNF) exposure within the last 90 days, recent hospitalization in the last 90 days with length of stay ≥ 2 days, prior intravenous antibiotic therapy or wound care in the last 30 days prior to admission, chronic HD, or a history of immunosuppression. Chronic HD persons were identified as those who have undergone dialysis at any setting plus a diagnosis of chronic kidney disease (based on ICD-9-CM codes) within 30 days prior to the index admission or during index admission. Based on clinical expertise, we classified immunosuppressed persons as those who had: 1) recent or current treatment with systemic corticosteroids or other immunosuppressive/chemotherapy agents; 2) a diagnosis of human immunodeficiency disease (HIV), leukaemia/lymphoma, or metastatic cancer, or 3) had undergone a solid/liquid organ transplant. All patients not fulfilling the HCAP definition were categorized as CAP.
Statistics and risk score development
For continuous variables, MRSA and non-MRSA patients were compared with either the Student’s t-test or Wilcoxon 2-sample test, as appropriate. Categorical data were analyzed with either the chi-square test or the Fisher’s exact test. We defined a p-value of <0.05 to represent statistical significance.
For risk score development and validation, we employed a split sample approach. The entire population was randomly split into a development cohort (two-thirds of patients) and a validation cohort (remaining one-third of patients). We then utilized a bootstrapping based, stepwise logistic regression procedure on the development cohort to determine variables most strongly associated with MRSA . We also assessed plausible interaction terms between various co morbidities as well as co morbidities and gender in a series of models. In each stepwise regression model, a p-value of 0.05 was required for entry into the model and a p-value of 0.1 to remain in the model. The stepwise algorithm was applied to 100 bootstrapped samples in the development set and each time the variable entered the model, this was recorded. Variables that consistently entered the model were kept for further analysis. The final 8 variables that comprised the risk score were those that entered the samples most often and had face validity for clinical relevance.
In the final risk score the standardized coefficients from the logistic regression represented the weight assigned to each predictor variable. Specifically, we fully standardized the coefficients of each variable in the final logistic regression model (i.e., they were standardized on the X and Y variables). Based on the standardized coefficients, we rounded up to get integer “point values” for the score with the smallest standardized coefficients getting a point value of 1 for ease of computation at the bedside. We then validated the risk score in the validation cohort, determining the prevalence of MRSA as a function of the score. We also examined traditional model performance characteristics (e.g., the area under the receiver operating characteristic curve [AUROC]).
(n = 837)
(n = 5138)
Age, mean ± SD, years
69.4 ± 17.1
67.1 ± 17.0
Co morbid illnesses (current or prior to index admission unless noted)
Diabetes mellitus, %
Coronary artery disease, %
Congestive heart failure, %
Chronic obstructive pulmonary disease, %
Malignancy (includes lymphoma, myeloma, secondary neoplasm), %
Cerebrovascular disease (any) prior to index admission, %
Ischemic stroke prior to index admission, %
Chronic haemodialysis, %
Healthcare-associated risk factors
Nursing home/skilled nursing facility/long term acute care exposure within last 90 days, %
Hospital-based wound care within 30 days prior to index admission, %
Recent hospitalization (for ≥ 2 days and within the last 90 days), %
Prior intravenous antibiotic therapy within the last 30 days, %
Severity of illness
Intensive care unit admission (on or before index culture), %
MRSA risk score for methicillin-resistant Staphylococcus aureus in pneumonia
Age < 30 years or > 79 years
Prior healthcare exposure
Recent hospitalization (for ≥ 2 days and within the last 90 days)
Nursing home/skilled nursing facility/long term acute care exposure within last 90 days
Prior IV antibiotic therapy within the last 30 days
Severity of illness
Intensive care unit admission (on or before index culture)
Co morbid illness
Cerebrovascular disease (any), prior to admission
Female with diabetes mellitus
Possible total point score: 10
As a screening test, a score of ≤1 versus greater than 1 had a sensitivity and specificity of 59.1% and 60.0% respectively. The positive predictive value was low at 19.2% but the negative predictive value equalled 90.1%. The pre-test probability for MRSA was approximately 14% (e.g., the prevalence of MRSA in the overall cohort). With a ≤1 classification threshold, patients with a score ≤1 would have a post-test (posterior) probability of MRSA of 10% (a reduction in risk of approximately one-third) and the post-test probability of MRSA would be 19.4% for those with scores >1. The AUROCs of this model in the development and validation cohorts were 0.66 (95% CI: 0.63, 0.68) and 0.64 (95% CI: 0.60, 0.67), respectively.
This large retrospective analysis of microbiologically-confirmed culture positive pneumonia patients presenting to the hospital reveals that MRSA constitutes an important pathogen in this setting. Similarly, HCAP accounts for a large proportion of all pneumonia patients admitted to the hospital and is nearly as common as CAP. Even though patients with HCAP face an increased risk for MRSA, the prevalence of MRSA in HCAP was less than 20%. A risk scoring tool based on factors independently associated with the recovery of MRSA segregates patients with moderate accuracy. A low total score describes a cohort of individuals unlikely to have MRSA and in whom anti-MRSA therapy can likely be withheld.
Our data add to the growing evidence elucidating the frequency of HCAP relative to CAP. In the initial study examining the epidemiology of HCAP, HCAP accounted for approximately one-third of all pneumonias presenting to the emergency department (ED) . Other reports have documented that HCAP represents between 30% and 60% of pneumonias initially evaluated in the ED [5–7], and this trend is also observed in Asia and Europe [9, 16]. Our study confirms that MRSA is a common pathogen in patients presenting to the hospital with their pulmonary infections. Classically thought only to be significant in nosocomial infections, it is now evident that physicians treating those who come to the hospital with pneumonia must consider MRSA when selecting antibiotic regimens.
Originally, the notion of HCAP was developed to help with this decision making process . The specific components of the HCAP definition were derived from expert opinion and were initially adopted from a single centre study in bacteraemia [3, 17]. HCAP was meant to serve as a tool to help clinicians stratify individuals as to the chance that a resistant pathogen, like MRSA, was a practical issue. Our finding that there are many patients with HCAP who are not infected with MRSA suggests that HCAP alone cannot serve as a precise tool for risk stratification. Our findings further confirm the observations of earlier researchers. For example, Schrieber and colleagues found that HCAP performed poorly as a screening test for MRSA in persons presenting to the hospital with severe pneumonia necessitating mechanical ventilation . Shorr et al. similarly found that HCAP misclassified many patients both as to their risk for resistant pathogens in general, and for MRSA, specifically . Thus, it appears that broad use of the HCAP criteria for classification can lead to overuse of broad spectrum antibiotics which can result in unnecessary cost and antimicrobial resistance.
The proposed risk score is novel and improves on the status quo, as it provides a means for reliably classifying quickly and easily a large cohort of patients at low risk for MRSA. Since anti-MRSA therapies can often be added or withheld without necessarily adjusting the selected Gram-negative coverage, having an effective way to limit the prescription of anti-MRSA treatments becomes crucial. The situation with respect to anti-MRSA treatment options is even more acute given their costs and potential side effects.
Prior reports have proposed alternate risk scoring schemes for determining the probability of resistant organisms in those coming to the hospital with pneumonia [5–7]. However, most of these studies have pooled resistant Gram negative organisms (e.g., P. aeruginosa) with MRSA [5–7]. Our effort is therefore unique in that we have examined MRSA specifically. Our results are also novel in that we identify several factors associated with MRSA that have not previously been linked to pneumonia with this pathogen. For example, extremes of age have not been previously described as related to MRSA in pneumonia. Likewise the description of a nexus between certain co morbid diseases, such as cerebrovascular disease (CVD) and dementia, with MRSA has not been noted before in evaluations of pneumonia in the ED. The mechanism of these associations with MRSA is unclear. These specific factors might actually represent surrogates for other factors that we could neither address nor measure in the dataset. Additionally, it appears that not all factors contribute equally to the risk for MRSA as a cause of pneumonia on hospital presentation. Intense exposure to the healthcare system (e.g., recent inpatient stay) seems disproportionately important as does severity of illness. Moreover, different risk scores utilized thus far in pneumonia (e.g., CURB-65) focus solely on severity of illness while our risk score specifically deals with issues of aetiology.
Our study has several strengths. It is based on a large sample from multiple hospitals across the US. This provides us the power to examine select conditions that other analyses could not evaluate because of their smaller size. It also suggests the generalizability of our findings. We have also internally validated the proposed risk score. No other effort has applied such a split sample approach in pneumonia presenting to the hospital. In fact, many risk stratification schemes for other disease states never undergo any effort at either internal or external validation.
Conversely, the present investigation has a number of significant limitations. First, its retrospective nature exposes it to several forms of bias. This is particularly concerning since we relied on ICD-9-CM coding to identify patients with pneumonia or a primary diagnosis of sepsis with a secondary diagnosis of pneumonia. There certainly are patients we included who likely had some alternate diagnosis. Second, culture data are not always obtained in every case of suspected pneumonia which introduces a selection bias. A priori, physicians may have preferentially opted to obtain cultures in patients when they were already concerned about resistant pathogens. Moreover, cultures can be negative even in the presence of active bacterial infection and positive cultures of the respiratory tract can represent colonization and not true infection. Again, this raises the concern that we could have misclassified patients. On the other hand, since we only studied culture positive patients, our findings likely underestimate the potential for antibiotic overuse and abuse. Third, we did not evaluate patients for co-infection with viral pathogens which could have influenced our findings, especially as a risk factor for MRSA infection. Fourth, despite our attempt to evaluate all components of the HCAP definition, we likely miscategorised some patients because of inadequate information regarding recent IV antibiotic exposure since administrative datasets may not consistently capture this. Some of the factors identified, such as dementia, could represent surrogates for other characteristics about which we lacked data. Finally, our score had only moderate discriminatory power and requires external validation. Although it identified a group of persons as low risk for MRSA, our data illustrate that there is a clear need for better tools for this purpose and that we urgently require rapid diagnostic tests for organism identification.
MRSA represents a cause of pneumonia presenting to the hospital. A simple risk score for identifying MRSA in patients presenting to the hospital with pneumonia identifies those at low risk for MRSA and may facilitate efforts to better direct antibiotic prescribing.
Area under the receiver operating curve
Chronic obstructive pulmonary disease
Congestive heart failure
Coronary artery disease
Health insurance portability and accountability act
Human immunodeficiency virus
Intensive care unit
International Classification of Diseases, 9th Revision, Clinical Modification
Institutional review board
Long term care facility
Methicillin resistant Staphylococcus aureus
Skilled nursing facility
This study was sponsored by Pfizer Inc.
Dr. Shorr was a paid consultant to Pfizer in connection with this study and the development of this manuscript. Dr. Emons is an employee of Cerner Corporation which received research funding from Pfizer in connection with conducting this study and development of this manuscript.
The authors wish to acknowledge Eileen Reyes of Cerner Corporation, Research Division for project management support, Hsing-Ting Yu of Cerner Corporation, Research Division for analytic support; and Richard Chambers of Pfizer Inc. for statistical support.
- Klevens RM, Morrison MA, Nadie J, Petit S, Gershman K, Ray S, Harrison LH, Lynfield R, Dumyati G, Townes JM, Craig AS, Zell ER, Fosheim GE, McDougal LK, Carey RB, Fridkin SK: Invasive Methicillin-resistant Staphylococcus aureus infections in the United States. JAMA. 2007, 298: 1763-1771. 10.1001/jama.298.15.1763.View ArticlePubMed
- Kallen AJ, Mu Y, Bulens S, Reingold A, Petit S, Gershman K, Ray SM, Harrison LH, Lynfield R, Dumyati G, Townes JM, Schaffner W, Patel PR, Fridkin SK: Healthcare-associated invasive MRSA infections, 2005–2008. JAMA. 2010, 304: 641-648. 10.1001/jama.2010.1115.View ArticlePubMed
- American Thoracic Society; Infectious Diseases Society of America: Guidelines for the management of adults with hospital-acquired, ventilator-associated, and healthcare-associated pneumonia. Am J Respir Crit Care Med. 2005, 171: 388-416.View Article
- Niederman MS: Community-acquired pneumonia: the U.S. perspective. Semin Respir Crit Care Med. 2009, 30: 179-188. 10.1055/s-0029-1202937.View ArticlePubMed
- Shorr AF, Zilberberg MD, Micek ST, Kollef MH: Prediction of infection due to antibiotic-resistant bacteria by select risk factors for health care-associated pneumonia. Arch Intern Med. 2008, 168: 2205-2210. 10.1001/archinte.168.20.2205.View ArticlePubMed
- Schreiber MP, Chan CM, Shorr AF: Resistant pathogens in nonnosocomial pneumonia and respiratory failure: is it time to refine the definition of health-care-associated pneumonia?. Chest. 2010, 137: 1283-1288. 10.1378/chest.09-2434.View ArticlePubMed
- Micek ST, Kollef KE, Reichley RM, Roubinian N, Kollef MH: Healthcare-associated pneumonia and community-acquired pneumonia: a single-centre experience. Antimicrob Agents Chemother. 2007, 51: 3568-3573. 10.1128/AAC.00851-07.PubMed CentralView ArticlePubMed
- Murphy CR, Quan V, Kim D, Peterson E, Whealon M, Tan G, Evans K, Meyers H, Cheung M, Lee BY, Mukamel DB, Huang SS: Nursing home characteristics associated with methicillin-resistant Staphylococcus aureus (MRSA) Burden and Transmission. BMC Infect Dis. 2012, 12: 269-10.1186/1471-2334-12-269.PubMed CentralView ArticlePubMed
- Shindo Y, Sato S, Maruyama E, Ohashi T, Ogawa M, Hashimoto N, Imaizumi K, Sato T, Hasegawa Y: Health-care-associated pneumonia among hospitalized patients in a Japanese community hospital. Chest. 2009, 135: 633-640. 10.1378/chest.08-1357.View ArticlePubMed
- Fine MJ, Auble TE, Yealy DM, Hanusa BH, Weissfeld LA, Singer DE, Coley CM, Marrie TJ, Kapoor WN: A prediction rule to identify low-risk patients with community-acquired pneumonia. NEJM. 1997, 336: 243-250. 10.1056/NEJM199701233360402.View ArticlePubMed
- Escobar GJ, Fireman BH, Palen TE, Gardner MN, Lee JY, Clark MP, Kipnis P: Risk adjusting community-acquired pneumonia hospital outcomes using automated databases. Am J Manag Care. 2008, 14: 158-166.PubMed
- Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR: Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001, 29: 1303-1310. 10.1097/00003246-200107000-00002.View ArticlePubMed
- Charlson ME, Pompei P, Ales KL, MacKenzie CR: A new method of classifying prognostic co morbidity in longitudinal studies: development and validation. J Chronic Dis. 1987, 40: 373-383. 10.1016/0021-9681(87)90171-8.View ArticlePubMed
- Austin PC, Tu J: Bootstrap Methods for Developing Predictive Models. Am Statistician. 2004, 58: 131-137. 10.1198/0003130043277.View Article
- Kollef MH, Shorr A, Tabak YP, Gupta V, Liu LZ, Johannes RS: Epidemiology and outcomes of health-care-associated pneumonia: results from a large US database of culture-positive pneumonia. Chest. 2005, 128: 3854-3862. 10.1378/chest.128.6.3854.View ArticlePubMed
- Venditti M, Falcone M, Corrao S, Licata G, Serra P, Study Group of the Italian Society of Internal Medicine: Outcomes of patients hospitalized with community-acquired, health care-associated, and hospital-acquired pneumonia. Ann Intern Med. 2009, 150: 19-26. 10.7326/0003-4819-150-1-200901060-00005.View ArticlePubMed
- Friedman ND, Kaye KS, Stout JE, McGarry SA, Trivette SL, Briggs JP, Lamm W, Clark C, MacFarquhar J, Walton AL, Reller LB, Sexton DJ: Healthcare-associated bloodstream infections in adults: a reason to change the accepted definition of community-acquired infections. Ann Intern Med. 2002, 137: 791-797. 10.7326/0003-4819-137-10-200211190-00007.View ArticlePubMed
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2334/13/268/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.