Description and validation of a spectrum score method to measure antimicrobial de-escalation in healthcare associated pneumonia from electronic medical records data
Comparison of antimicrobial de-escalation rates between healthcare settings is problematic. To objectively and electronically measure de-escalation a method based upon the spectrum of antimicrobial regimens administered (i.e., spectrum score) was developed.
Methods
A Delphi process was used to develop applicable concepts. Spectrum scores were created for 27 antimicrobials based upon susceptibility for 19 organisms. National VA susceptibility data was used to estimate microbial spectrum. Susceptibility estimates were converted to ordinal scores, and values for organisms with multi-drug resistance potential were weighted more heavily. Organism scores were summed to create antibiotic-specific spectrum scores and extended mathematically to score multi-antimicrobial regimens. Vignettes were created from antimicrobial regimens administered to 300 patients hospitalized with pneumonia. Daily spectrum scores were calculated for each case. Hospitalization day 4 scores were subtracted from day 2 scores (i.e., spectrum score ∆). A positive spectrum score ∆ defined de-escalation. Experts ranked each pneumonia case on a 7-point Likert scale (Likert >4 indicated de-escalation). Spectrum score ∆s were compared to expert review. Findings were used to identify score deficiencies. Next, 40 pairs of cases were modified to include antimicrobial administration routes. Each pair contained almost similar regimens; however, one contained oral (PO) the other only intravenous (IV) antimicrobials on day 4 of therapy. Experts reviewed cases as described. Spectrum score ∆ credits to account for PO conversion were derived from the mean paired differences in Likert Score. De-escalation status was evaluated in 100 vignettes containing antimicrobial route by different experts and compared to the modified method.
Results
Initial sensitivity and specificity of the spectrum score ∆ to detect expert classified de-escalation events was 86.3 and 96.0%, respectively. In paired cases, the mean (± SD) Likert score was 5.0 (1.5) and 4.6 (1.5) for PO and IV (P = 0.002), respectively. To improve de-escalation event detection, two credits were added to spectrum score ∆s based upon the percentage of antimicrobials administered PO on day 4. The final method, exhibited sensitivity and specificity to detect expert classified de-escalation events of 96.2 and 93.6%, respectively.
Conclusions
The final spectrum score method exhibited excellent agreement with expert judgments of de-escalation events in pneumonia.
In response to increasing antimicrobial resistance, professional and governmental organizations recommend that hospital-based programs practice antimicrobial stewardship [1-3]. Antimicrobial stewardship refers to coordinated interventions to optimize the selection, dose, and duration of antimicrobial therapy, while limiting adverse events, cost, and antimicrobial resistance. Antimicrobial de-escalation is a fundamental operation of antimicrobial stewardship [1-5].
Antimicrobial de-escalation refers to a reduction in the antimicrobial “spectrum” of therapy, through discontinuation of antimicrobials providing activity against non-pathogenic flora or changing antimicrobials to limit coverage to targeted pathogens once a patient is clinically stable [1-5]. Stopping antimicrobials with overlapping spectrum or, discontinuing antimicrobials if an infection is unlikely may also be considered antimicrobial de-escalation [5,6]. In studies, de-escalation has been defined and measured subjectively based upon variable qualitative assessments [5]. Further, manual chart review is required to determine if antimicrobial de-escalation has occurred. Comparison of antimicrobial de-escalation practice is limited due to a lack of objective measurement criteria and automated measurement methods.
We developed a method to measure antimicrobial de-escalation, which was based upon the spectrum of antimicrobial regimens administered that can be applied to electronic medical records data (i.e., spectrum score method). The methodological framework was based upon the opinions of antimicrobial stewards for concepts relevant to antimicrobial de-escalation, which were obtained through a Delphi process. The method was developed to estimate facility-level de-escalation rates with data from the U.S. Veterans Healthcare Administration (VA) Computerized Patient Record System. A description of the Delphi panel findings and a preliminary overview of the spectrum score method have been reported [7].
The purpose of this manuscript is to provide a detailed description of the spectrum score method, including: the approach to assignment of spectrum scores to antimicrobials, application of the scores to electronic medical records data to generate spectrum scores for antimicrobial regimens; refinement and validation of the method to measure antimicrobial de-escalation in patients with Healthcare Associated Pneumonia (HCAP). HCAP was selected for evaluation because broad-spectrum empirical antimicrobial therapy and de-escalation are indicated [8].
Methods
The spectrum score method involves assignment of a numerical score to each calendar day of antimicrobial therapy administered during hospitalization based on the microbial spectrum of the regimen. Antimicrobial de-escalation is quantified by subtracting the regimen’s spectrum score for day 4 of hospitalization from a baseline score obtained for antimicrobials administered on day 2 of hospitalization (i.e., spectrum score ∆). A positive spectrum score ∆ suggests that microbial spectrum coverage has been narrowed and that de-escalation has occurred.
Assignment of spectrum score to antimicrobials
The approach to assigning a numerical spectrum score to antimicrobial regimens has recently been summarized [7]. First, national VA Corporate Data Warehouse (CDW) susceptibility data for organisms and antimicrobials tested during years 2008–2012 were used to estimate antimicrobial susceptibility if available. The VA CDW includes culture and susceptibility results for 7 million veterans who receive care in 152 medical centers. Percent susceptibility was calculated for individual antimicrobial-organism pairs utilizing one isolate per patient per year. Next, remaining organism-antimicrobial pairs were categorized as possessing no intrinsic activity (NA), no VA susceptibility data available (ND), or further confirmation of susceptibility estimates required. NA classifications were based upon current FDA approved prescribing information, Clinical Laboratory Standards Institute (CLSI) standards for susceptibility testing, and tertiary references [9-42]. Current CLSI reporting recommendations were applied preferentially. For example, the CLSI-based susceptibility of Staphylococcus aureus to oxacillin was used to populate the susceptibility values for other β-lactams (except ceftaroline) with activity against Staphylococcus aureus. Organism-antimicrobial pairs without approved susceptibility methods, but suppressed results, were handled individually; where possible findings were cross-referenced with susceptibility results identified through literature sources.
Assignment of scores to organism-antimicrobial pairs without susceptibility data was directed by several data sources, and in some cases more than one data source was used. Primary literature was evaluated for susceptibility studies. Preference was given to results from studies which included U.S. based isolates, referenced CLSI methods, and recent studies. In some cases, CLSI documents provided estimates of susceptibility, in other cases, if recent product labeling indicated that the antimicrobial possessed in vitro activity against > 90% of the isolates tested in clinical trials, a value of 90% was assigned. Infrequently, suitable references could not be identified, and investigator opinion was utilized to assign susceptibility. To enhance generalizability and apply susceptibility data to be more representative of different practice environments, percentage susceptibilities were converted into quintiles ranging from 0 points for susceptibilities of < 20% to 4 points for susceptibility of 80-100%. Assignment of ordinal scores to organism-antimicrobial pairs without VA susceptibility data was performed independently by two investigators (KM, BH) with adjudication of discrepancies by a 3rd investigator (MJ).
Spectrum score adjustment for intrinsically resistant organisms
Delphi panelists indicated that antimicrobials covering organisms with high potential for developing resistance should receive extra weight in the spectrum score [7]. Consistent with panelist preferences, ordinal susceptibility values for organism-antimicrobial pairs involving Staphylococcus aureus, Enterococcus faecium, Escherichia coli, Klebsiella spp., and Acinetobacter spp. were multiplied by a factor of 1.25, and Pseudomonas aeruginosa was multiplied by a factor of 1.75. Domains for each organism’s weighted or un-weighted scores were added to create a composite spectrum score for each antimicrobial on a 0–60 spectrum of activity scale.
Calculation of spectrum scores for combination antimicrobial therapy
To account for overlapping coverage in combination regimens, the following approach was taken. First, if none of the regimen antimicrobials possessed activity against a species, a zero was assigned to the organism-combination regimen pair. If one of the regimen antimicrobials possessed activity against a species, the active antimicrobial susceptibility was used to populate the organism combination regimen pair. For other regimens where a combination of antimicrobials possessed activity against a species, the proportion of organisms susceptible to a multi-antimicrobial regimen was estimated as one minus the joint probability of resistance to all regimen antimicrobials, assuming that antibiotic susceptibilities were independent [7]. Spectrum scores were then computed identically to individual antimicrobial regimens. Table 1 illustrates the steps involved in generating spectrum score values for an individual and combination antimicrobial regimens.
Table 1
Conceptual illustration of spectrum score calculation for an antimicrobial regimen
Step 1: Populate susceptibility percentage for organism domains, convert values to ordinal scale (ordinal values shown in parentheses)A,B,C,D
Organism domain
Vancomycin susceptibility (%)
Vancomycin ordinal score
Cefepime susceptibility (%)
Cefepime ordinal score
Vancomycin + cefepime joint susceptibilityB
Vancomycin + cefepime joint ordinal score
Staphylococcus aureus
99.3
4
51.2
2
99.6
4
Streptococcus pneumoniae
98.2
4
92.5
4
99.9
4
Enterococcus faecium
18.4
0
NA
0
18.4
0
Enterococcus faecalis
94.7
4
NA
0
94.7
4
Escherichia coli
NA
0
92.4
4
92.4
4
Klebsiella spp.
NA
0
84.5
4
84.5
4
Other enterobacteriaceaeC
NA
0
91.3
4
91.3
4
Pseudomonas aeruginosa
NA
0
78.0
3
78.0
3
Acinetobacter spp.
NA
0
42.6
2
42.6
2
Stenotrophomonas spp.
NA
0
36.6
1
36.6
1
Haemophilus influenzae
NA
0
95.4
4
95.4
4
Bacteroides spp.
NA
0
NA
0
NA
0
Legionella spp.
NA
0
NA
0
NA
0
Mycoplasma spp.
NA
0
NA
0
NA
0
Step 2: Weight susceptibilities for coverage against intrinsically resistant organisms, sum organism domain scores to create antibiotic regimen spectrum scoreE
Organism domain
Vancomycin weighted ordinal score
Cefepime weighted ordinal score
Vancomycin + cefepime weighted ordinal score
Staphylococcus aureus
5
2.5
5
Streptococcus pneumoniae
4
4
4
Enterococcus faecium
0
0
0
Enterococcus faecalis
4
0
4
Escherichia coli
0
5
5
Klebsiella spp.
0
5
5
Other enterobacteriaceaeC
0
4
4
Pseudomonas aeruginosa
0
5.25
5.25
Acinetobacter spp.
0
2.5
2.5
Stenotrophomonas spp.
0
1
1
Haemophilus influenzae
0
4
4
Bacteroides spp.
0
0
0
Legionella spp.
0
0
0
Mycoplasma spp.
0
0
0
Spectrum Score
13.0
33.25
39.75
Spectrum Score Calculations for Individual and Combination Antibiotic Regimens. AValues populated with susceptibility data. Susceptibility estimates for combinations where all antimicrobials possessed activity against the species obtained by calculating one minus the joint probability of resistance to all antibiotics in the regimen, assuming that susceptibility was independent for each antibiotic. BOther enterobacteriaceae included: Citrobacter spp., Enterobacter spp., Morganella spp., Proteus spp., Providencia spp., Serratia spp. CNA = No intrinsic activity. DOrdinal values were 0 for no intrinsic bacterial activity or susceptibility < 20%, 1 for > 20 but < 40%, 2 for >40 but < 60%, 3 for >60 but < 80%, 4 > 80 %. EA weight of 1.25 was applied to ordinal domain values for Staphylococcus aureus, Escherichia coli, Klebsiella spp., Acinetobacter spp., Enterococcus faecium, and a weight of 1.75 was applied to spectrum score values for Pseudomonas aeruginosa.
Application of the spectrum score method to measure de-escalation
Inpatient antimicrobial administration within the VA is documented utilizing Bar Code Medication Administration (BCMA) technology. For each administered dose of antimicrobial, data regarding the dose and route of administration are recorded electronically with a time stamp [43]. Antimicrobial use data was obtained for a VA-wide cohort (years 2008–2012) of inpatient admissions with HCAP [8,44]. Each systemically administered antimicrobial the patient received during each calendar day of admission was extracted from the CDW [45]. As combination antimicrobial therapy was commonly administered, data were “smoothed” to prevent over and under-calculation of spectrum scores on days when different antimicrobials were simultaneously being initiated or discontinued. Smoothing was accomplished by inferring the intent to treat from antimicrobial administration data. Skip days, where antibiotics that were administered less frequently than once daily, were filled when between adjacent calendar days where the same antibiotic was administered (e.g., vancomycin administered every 48 hours). Skip days were assigned the same antibiotic regimen as the adjacent calendar days. To avoid double counting antibiotics when an antibiotic regimen was switched, an antibiotic day was not counted when there was a change in a multidrug regimen from the current, previous, and next days, the antibiotic was given in the past two days but not in the next three, and the day did not fall on the beginning or end of an antibiotic treatment course. For example, switching from cefepime and vancomycin to levofloxacin and clindamycin on day 4 would appear as a four-drug regimen unless these rules are invoked, in which case only the latter regimen would be counted (Table 2).
Table 2
Example of a prediction of antimicrobial de-escalation status by the spectrum score method
A. Daily antimicrobial (Route of administration) use data
Antimicrobial/Day
Hospital day 1
Hospital day 2
Hospital day 3
Hospital day 4
Hospital day 5
Clindamycin (Clm)
Clm (PO)
Clm (PO)
Levofloxacin (Lev)
Lev (PO)
Lev (PO)
Cefepime (Cpm)
Cpm(IV)
Cpm(IV)
Cpm (IV)
Cpm (IV)
Vancomycin (Vm)
Vm (IV)
Vm (IV)
Vm (IV)
Vm (IV)
Daily Regimen Spectrum Score
39.75
39.75
39.75
55.00
44.25
B. Application of “smoothing” rules to daily antimicrobial use data
Antimicrobial/Day
Hospital day 1
Hospital day 2
Hospital day 3
Hospital day 4
Hospital day 5
Clindamycin (Clm)
Clm (PO)
Clm (PO)
Levofloxacin (Lev)
Lev (PO)
Lev (PO)
Cefepime (Cpm)
Cpm(IV)
Cpm (IV)
Cpm (IV)
Vancomycin (Vm)
Vm (IV)
Vm (IV)
Vm (IV)
Daily Regimen Spectrum Score
39.75
39.75
39.75
44.25
44.25
C. Calculation of spectrum score ∆ (Day 2 spectrum score – Day 4 spectrum score)
[Cefepime + vancomycin score]
–
[Clindamycin + levofloxacin score]
=
Spectrum score ∆
39.75
–
44.25
=
-4.5
D. Adjustment of spectrum score ∆ for PO antimicrobials administered on day 4
Spectrum score ∆
+
Administration credit
Final spectrum score ∆ after PO adjustment
-4.5
+
6
1.5
A. Example of antimicrobials administered on a daily basis during a 5 day hospitalization. Days 2 and 4 indicate baseline and de-escalation determination endpoints. Note that the daily antimicrobial regimen spectrum score increases on day 4 due to the addition of oral antimicrobials; however, IV antimicrobials are discontinued. B. Smoothing rules were applied to the daily antimicrobial administration data which results in a lower day 4 spectrum score. C. The spectrum score ∆ is negative suggesting that the spectrum of activity of is greater for the clindamycin + levofloxacin regimen than for the cefepime + vancomycin combination. D. The PO credit is applied to the case because 100% of the antimicrobials on day 4 were administered PO (6 points for regimens with ≥ 50 % of antimicrobials administered PO by day 4). The final spectrum score ∆ with PO credit is positive prediciting that a de-escalation event occurred.
Spectrum score method refinement and validation
Refinement and validation of the spectrum score method was performed in three stages. First, 300 vignettes were created based on daily antimicrobial regimens obtained from a random sample of patients who met HCAP criteria [8,44]. Vignettes included antimicrobials administered on each calendar day of hospitalization and microbiology findings obtained in the first two days. Three antimicrobial stewards [mean (SD) 17.3 (0.7) years of experience] who were unfamiliar with the spectrum score method reviewed the vignettes. The stewards ranked each case on a 7-point Likert scale: de-escalation (score >4), no meaningful change in therapy (score =4), or escalation (score of < 4). Spectrum scores were determined for regimens administered on calendar days 2 and 4 of hospitalization; day 4 scores were subtracted from day 2 scores resulting in a spectrum score ∆ for each vignette. A positive spectrum score ∆ indicated de-escalation. Sensitivity and specificity of the sign of spectrum score ∆ to predict antimicrobial steward de-escalation opinion (i.e., reference standard) were calculated. To determine if varying the weight for coverage of resistant organisms impacted test characteristics of the method, spectrum score variants with differing weights (no weight, 1.25, 1.5, 1.75, 2.0, 3.0) for coverage of these organisms were calculated for the cases, and then compared to expert opinion in a similar manner.
Based upon observations in the initial exercise, 40 pairs of vignettes were modified to include the administration route for each antimicrobial. Each pair of vignettes contained identical or almost identical antimicrobial regimens with the same spectrum; however, in one case the regimen on day 4 contained one or more antimicrobials administered orally (PO) whereas in the other case all antimicrobials were administered intravenously (IV) (i.e. amoxicillin versus ampicillin). These cases were evaluated by the same experts and compared to the spectrum score ∆ as described. Mean paired difference in Likert scores between the IV and PO cases was calculated for pairs in which the PO case had 50-100% PO therapy and for pairs with >0% but less than 50% PO therapy on day 4. A linear regression model relating mean Likert and spectrum score Δ for vignettes with 100% IV therapy on day 2 and day 4 was used to express the means of paired difference in Likert in spectrum score scale (paired Likert means divided by the slope). The mean values converted to the spectrum score scale provided additional credits to the spectrum score Δ to account for vignettes with minimal conversion to PO therapy (>0% but <50%) and greater or full conversion to PO therapy (50%-100%) on day 4. Finally, 100 vignettes were selected from the original 300 HCAP cases and the route of antimicrobial administration data was added. These vignettes were assessed for de-escalation status by three new antimicrobial stewards unfamiliar with the spectrum score method [mean (SD) 14.0 (5.2) years of experience] using the same 7-point Likert scale. Mean Likert scores were compared to the spectrum score ∆ credited for IV to PO conversion. Only vignettes with 100% IV therapy on day 2 were eligible for the credit. Test characteristics of the spectrum score method to predict antimicrobial de-escalation were performed as above. Table 2 illustrates application of the spectrum score method to measure de-escalation in a theoretical case, and Figure 1 summarizes the process for development, refinement, and validation of the spectrum score method. A procedure that can be used to apply the spectrum score method is available in Additional files 1 and 2.
Figure 1
Overview of the process for development, refinement, and validation of the spectrum score method.
This research complies with all Federal guidelines and VA policies relative to Human Subjects and research. The VA Puget Sound Healthcare System, University of Utah, and Idaho State University Human Subjects Committees approved the study and its conduct was in compliance with the Helsinki Declaration.
Results
Spectrum score calculations
Spectrum scores were determined for 14 organism domains which included 19 bacterial species and 27 categories of antimicrobials. Table 3 illustrates the ordinal susceptibility scores for organism-antimicrobial pairs. There were 513 possible organism-antimicrobial pairs for consideration of which 53% had sufficient VA susceptibility data, 6% were determined by applying CLSI-based rules, 28% were identified as possessing NA, and 15% required further confirmation or determination of susceptibility results with external sources. Suitable references could not be identified in a 3% of cases and domain scores were assigned by investigator opinion.
Table 3
Ordinal susceptibility scores for antimicrobial-organism pairs included in the spectrum score
******Miscellaneous: PSAFQ = ciprofloxacin or levofloxacin; NONPSAFQ = moxifloxacin, gemifloxacin; Amik = Amikacin; Gent or Tobra = gentamicin, tobramycin; Macro = erythromycin, azithromycin, clarithromycin; Clinda = clindamycin; Tetra = tetracycline, doxycycline; TMP/SMX = trimethoprim/sulfamethoxazole; Metro = metronidazole; Colistin = colistin.
Spectrum score method refinement and validation
Findings for the initial validation exercise have recently been overviewed [7]. Briefly, there were 142 distinct antimicrobial regimens (79% combination regimens) administered in the 300 HCAP patient cases. Day 2 and 4 spectrum scores in the vignettes ranged from a minimum of 4.0 to a maximum of 60.0. The sensitivity and specificity of the spectrum score method to identify antimicrobial de-escalation events as determined by antimicrobial stewards was 86.3 and 96.0%, respectively. Adjustment of weights assigned for coverage of intrinsically resistant organisms did not improve the sensitivity and specificity of the spectrum score method beyond the weighting proposed by the Delphi panel, and spectrum score ∆ was more predictive of de-escalation events with application of these weights (P = 0.05, difference in area under the curve between weighted and un-weighted receiver operator curves). Upon inspection, it appeared that in select vignettes stewards inferred that a patient received a PO antimicrobial on day 4 even though the route of antimicrobial administration was not stated (e.g., ceftriaxone administered on day 2 followed by cefpodoxime administered on day 4). Despite many of these regimens possessing similar spectrum scores (i.e., spectrum score for both ceftriaxone or cefpodoxime was 25.25) , experts scored these vignettes higher if regimens contained antimicrobials with PO dosage forms available compared to regimens with IV antibiotics on day 4 (e.g., ceftriaxone on day 2 and day 4).
In the second vignette exercise, the mean (± SD) Likert score for paired vignettes containing identical or nearly identical antimicrobials with the same spectrum but differing routes of administration by day 4 was 5.0 (1.5) and 4.6 (1.5) for PO and IV cases (P = 0.002), respectively. The mean (± SD) paired difference in Likert scores was 0.44 (0.50) for pairs where the percentage of PO therapy in the PO case was 50-100% by day 4 (P < 0.001) and 0.24 (0.68) for pairs where the percentage of PO therapy was > 0% but < 50% in the PO case by day 4 (P = 0.12). Based on the linear relationship between mean Likert score and weighted spectrum score ∆ (slope = 0.072, P < 0.001) for vignettes with 100% IV therapy on day 2 and day 4, the paired mean difference in Likert score for vignettes with 50-100% PO therapy on day 4 was equivalent to approximately 6 spectrum score points, and was equivalent to approximately 3 spectrum score points for >0% but <50% PO therapy on day 4. Adding the two credits to spectrum score ∆s for applicable regimens improved detection of de-escalation events in the set of paired vignettes as classified by stewardship experts.
In the final exercise, the sensitivity and specificity of the spectrum score method, including the credit for PO antimicrobials to predict de-escalation events was 96.2 (95% CI >88.6%) and 93.6% (95% CI >84.3%), respectively. Antimicrobial stewards identified de-escalation, no meaningful change in therapy, or escalation in 53, 41, and 6% of vignettes reviewed whereas, the spectrum score method with credits identified de-escalation, no meaningful change in therapy, or escalation by day 4 of therapy in 54, 42, and 4 of these vignettes, respectively. Agreement between reviewers was high [Mean (±95% CI) intra-class correlation coefficient 0.86 (0.81-0.91)]. The sensitivity of the spectrum score method without PO credits to predict steward judgments of de-escalation events was only 56.7% (specificity 97.9%). Vignettes included in the final exercise were created as a subset of the initial 300 cases, and comparison of the mean (±SD) Likert scores for vignettes included in both the initial and final and exercises was 4.2(1.3) and 4.6 (1.1), respectively (P <0.003).
Taken together the vignette exercises suggest that inclusion of the route of administration was an important consideration in the assessment of de-escalation events, and that IV to PO conversion was viewed favorably when assessing de-escalation events. Table 4 summarizes the three refinement and validation exercise findings.
Table 4
Validation and refinement of the spectrum score method to measure antimicrobial de-escalation
Validation exercise
Sensitivity (%)
Specificity (%)
Positive predictive value (%)
Negative predictive value (%)
Comments
Convergent validation of original spectrum score
86.3
96.0
87.5
95.6
Reference standard de-escalation prevalence in vignettes was 24.2%. Route of administration data not included in cases. Mean Likert score was 0.51 points higher for cases where regimens contained ≥ 1 antimicrobial available in a PO dosage form administered on day 4 (p = 0.003).
Refinement exercise to verify that IV to PO conversion impacted expert opinion of de-escalation events
NA
NA
NA
NA
Mean Likert scores for regimens containing similar antimicrobials but differing routes of administration by day 4 was 5.0 (1.5) and 4.6 (1.5) for PO and IV cases (P = 0.002), respectively. Linear regression used to estimate an additional credit to the spectrum score ∆ to account for the group mean differences in Likert score associated with conversion from IV to PO therapy. A 6 point credit was added to spectrum score ∆ values for regimens with >50-100% PO and 3 point credit for regimens >0 but < 50% PO.
Convergent validation of spectrum score method including PO offsets
96.2
93.6
94.4
95.7
Reference standard de-escalation prevalence in vignettes was 53.0%. Route of administration data included in cases and PO credits applied to spectrum score ∆.
There were five discordant cases between the spectrum score with PO credits and reference method. Tetracyclines, which were only included in five vignettes, were involved in three of the discordant cases. Reviewers scored regimens in favor of de-escalation when tetracyclines were included on day 4 compared to the spectrum score method. Another discordant case involved the addition of a duplicative spectrum PO antibiotic to an IV regimen on day 4, which was classified an escalation by reviewers and a de-escalation by the spectrum score method due to the PO credit awarded. In the final discordant case, one of four antimicrobials was administered PO on day 2, but the antimicrobials remained identical on day 4 with two antimicrobials administered by the PO route. Experts scored this case as a de-escalation; however the case was not eligible for the credit as not all antimicrobials were administered IV on day 2.
Discussion
The spectrum score method and findings are unique, and we are unaware of studies where definitions of antimicrobial de-escalation have been compared. Development of this approach was based in part on the opinions of antimicrobial stewards obtained during the Delphi process which enhanced construct validity. In the absence of a clear reference standard we compared the spectrum score method to that of antimicrobial stewards who have the most expertise and interest in the measurement of de-escalation events. A major finding of this investigation includes the high degree of agreement between the spectrum score predictions of de-escalation status and independent expert judgments applied at a patient-level which suggests convergent validity. A second major finding is the observation that antimicrobial stewards view IV to PO conversion favorably when classifying de-escalation events. Frequently clinicians make pragmatic tradeoffs when altering antimicrobial therapy in preparation for discharge which sometimes may be viewed as therapy simplification rather than verbatim selection of the most narrow-spectrum antimicrobial. While addition of the PO credits improved the sensitivity to detect de-escalation events classified by stewards, providing information on the route of antimicrobial administration to the experts in addition to the specific antimicrobials administered resulted in an increase in classification of de-escalation events in the vignettes. Further, the sensitivity to detect de-escalation events based on spectrum alone decreased when the route of administration information was added to the cases. The observation that stewards perceived tetracyclines as a relatively narrow-spectrum antimicrobial class despite susceptibility data to the contrary is also interesting. The reason behind this perception is unclear but may be related to the use of tetracyclines almost exclusively for coverage of atypical pathogens in treatment of pneumonia within the U.S., or a (subconscious) tendency to consider use of older drugs as a form of de-escalation.
Strengths of the method are that it was developed based upon the opinions of antimicrobial stewards and is anchored within the constructs of objective VA-wide microbial susceptibility data. The decision points of hospital day 2 and day 4 to measure baseline and follow-up therapy were supported as favorable time-points for measuring de-escalation events by antimicrobial stewards [7,46]. An additional strength of the method is that it can be adapted to classify antimicrobial de-escalation events in other electronic medical record systems.
Limitations of the method include reliance upon VA microbiology data to define spectrum score values. Antimicrobial susceptibilities and hence spectrum scores will likely change over time; which will require a regular re-evaluation of organism-antimicrobial susceptibilities. Further, in some instances VA susceptibility data were lacking and published susceptibility estimates were dated. It is possible that differences in susceptibility patterns across facilities may impact drug selection and de-escalation practice; however in an attempt to minimize these differences, we converted susceptibility estimates to ordinal values to facilitate generalization and organisms with MDR potential were weighted more heavily in accordance with Delphi panel preferences. The number of antimicrobial stewardship experts rendering judgments on de-escalation events was limited, and consensus was not obtained on all patient cases. In the final validation exercise, sensitivity of the method to detect de-escalation events in the vignettes was highly dependent on the addition of PO credits. It is important to remember that the estimates for de-escalation were generated from vignettes which were based upon antimicrobial regimens administered to patients in the cohort. In limited cases it was necessary to makes slight modifications of regimens to fit the vignette format, and it is unknown if the observed de-escalation rate reflects the actual de-escalation rate as measured in patients. However, the importance of PO conversion was identified in all three validation and refinement exercises. Conversely, disagreements between the spectrum score method and the expert judgments are to be expected, and lack of 100% agreement is not evidence for lack of validity.
Antimicrobial de-escalation and IV to PO conversion programs have been associated with reductions in length of hospitalization, inpatient antimicrobial use, adverse events, cost, and recovery of antimicrobial-resistant microbes [1,5,47-49]. However, few studies have clearly defined how de-escalation status was measured, and the quality of evidence in this area is poor [5,50]. The use of a limited scoring system based upon Gram-negative activity of anti-pseudomonal β-lactam and fluoroquinolone antibiotics has been previously described [51]. Recently, quality indicators for the management of antimicrobial use in sepsis have been described that recommend “changing to pathogen-directed therapy after culture-results become available” [52]. An IV to PO conversion quality measure has also been proposed for highly bioavailable antimicrobials in a clinically stable population; however we are unaware of recommendations that incorporate both concepts into a single measure [53].
Future work should include application and automation of the spectrum score method to measure antimicrobial de-escalation in electronic medical records. We are currently conducting such an analysis of VA-wide facility-level de-escalation rates in HCAP. Additional work should also include an assessment of the importance of PO therapy in the assessment of de-escalation therapy. Further, validation of the spectrum score method to estimate antimicrobial de-escalation in other patient populations, disease states, and electronic medical records systems is warranted.
Conclusions
The spectrum score method exhibited excellent agreement with antimicrobial steward judgments of antimicrobial de-escalation events in pneumonia. The method, which is based upon the spectrum of antimicrobial regimens administered at subsequent time-points during hospitalization and the utilization of IV to PO conversion, can be applied to electronic medical records data to assess antimicrobial de-escalation in patients with healthcare associated pneumonia.
Abbreviations
Spectrum Score ∆:
Subtraction of the day 4 spectrum score value from the day 2 spectrum score value
PO:
Oral
IV:
Intravenous
VA:
Veterans Healthcare Administration
HCAP:
Healthcare associated pneumonia
NA:
No intrinsic activity
ND:
No VA susceptibility data available
CLSI:
Clinical Laboratory Standards Institute
BCMA:
Bar code medication administration
SD:
Standard deviation 95%
CI:
Ninety-five percent confidence intervals
Declarations
Acknowledgements
This work was supported in part, with resources and use of the Boise and George E. Wahlen Veterans Affairs Medical Centers. The views expressed in this article are solely those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs.
Authors’ Affiliations
(1)
Pharmacy Service, Boise Veterans Affairs Medical Center and College of Pharmacy, Idaho State University
(2)
IDEAS Center, VA Salt Lake City Health Care System and Division of Epidemiology Health Care System and Division of Epidemiology, University of Utah
(3)
Research Service, Boise Veterans Affairs Medical Center and Quantified Inc
(4)
Research Fellow, Pharmacy Service, Boise Veterans Affairs Medical Center
(5)
Infection Control Programme & Division of Infectious Diseases, Faculty of Medicine, Geneva University Hospitals
(6)
IDEAS Center, VA Salt Lake City Health Care System and Division of Epidemiology, University of Utah
References
Dellit TH, Owens RC, McGowan Jr JE, Gerding DN, Weinstein RA, Burke JP, et al. Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America guidelines for developing an institutional program to enhance antimicrobial stewardship. Clin Infect Dis. 2007;44(2):159–77.View ArticlePubMedGoogle Scholar
Fishman N, Patterson J, Saiman L, Srinivasan A, Trivedi KK, van Schooneveld T, et al. Policy statement on antimicrobial stewardship by the Society for Healthcare Epidemiology of America (SHEA), the Infectious Diseases Society of America (IDSA), and the Pediatric Infectious Diseases Society (PIDS). Society for Healthcare Epidemiology of America; Infectious Diseases Society of America; Pediatric Infectious Diseases Society. Infect Control Hosp Epidemiol. 2012;33(4):322–7.View ArticleGoogle Scholar
Deresinski S. Principles of antibiotic therapy in severe infections: optimizing the therapeutic approach by use of laboratory and clinical data. Clin Infect Dis. 2007;45 Suppl 3:S177–83.View ArticlePubMedGoogle Scholar
Kollef MH. Providing appropriate antimicrobial therapy in the intensive care unit: surveillance vs. de-escalation. Crit Care Med. 2006;34(3):903–5.View ArticlePubMedGoogle Scholar
Madaras-Kelly KJ, Jones M, Remington R, Hill N, Huttner B, Samore M. Development of an antibiotic spectrum score based on veterans affairs culture and susceptibility data for the purpose of measuring antibiotic de-escalation: a modified Delphi approach. Infect Control Hosp Epidemiol. 2014;35(9):1103–13.View ArticlePubMedPubMed CentralGoogle Scholar
Anon. American Thoracic Society; Infectious Diseases Society of America. Management of adults with hospital-acquired, ventilator-associated, and healthcare-associated pneumonia. Am J Respir Crit Care Med. 2005;171(4):388–416.View ArticleGoogle Scholar
Anon. Clinical Laboratory Standards Institute. Performance Standards for Antimicrobial Susceptibility Testing. 2012: M100-S22; M076-A9Google Scholar
Anon. Clinical Laboratory Standards Institute. Methods for dilution of antimicrobial susceptibility tests for bacteria that group aerobically; 2012. p. M07-A9Google Scholar
Anadiotis L, Maskell JP, Sefton AM. Comparative in-vitro activity of penicillin alone and combined with gentamicin against clinical isolates of Streptococcus pneumoniae with decreased susceptibility to penicillin. Int J Antimicrob Agents. 2002;19(3):173–81.View ArticlePubMedGoogle Scholar
Asbell PA, Colby KA, Deng S, McDonnell P, Meisler DM, Raizman MB, et al. Ocular TRUST: nationwide antimicrobial susceptibility patterns in ocular isolates. Am J Ophthalmol. 2008;145(6):951–8.View ArticlePubMedGoogle Scholar
Biedenbach DJ, Jones RN. In vitro activity of linezolid (U-100766) against Haemophilus influenzae measured by three different susceptibility testing methods. Diagn Microbiol Infect Dis. 2001;39(1):49–53.View ArticlePubMedGoogle Scholar
Blondeau JM, Yaschuk Y, Suter M, Vaughan D. In-vitro susceptibility of 1982 respiratory tract pathogens and 1921 urinary tract pathogens against 19 antimicrobial agents: a Canadian multicentre study. Canadian Antimicrobial Study Group. J Antimicrob Chemother. 1999;43 Suppl A:3–23.View ArticlePubMedGoogle Scholar
Bouchillon SK, Hoban DJ, Johnson BM, Johnson JL, Hsiung A, Dowzicky MJ. In vitro activity of tigecycline against 3989 Gram-negative and Gram-positive clinical isolates from the United States Tigecycline Evaluation and Surveillance Trial (TEST Program; 2004). Diagn Microbiol Infect Dis. 2005;52(3):173–9.View ArticlePubMedGoogle Scholar
Citron DM, Goldstein EJ, Kenner MA, Burnham LB, Inderlied CB. Activity of ampicillin/sulbactam, ticarcillin/clavulanate, clarithromycin, and eleven other antimicrobial agents against anaerobic bacteria isolated from infections in children. Clin Infect Dis. 1995;20 Suppl 2:S356–60.View ArticlePubMedGoogle Scholar
De Fátima Silva Lopes M, Ribeiro T, Abrantes M, Figueiredo Marques JJ, Tenreiro R, Crespo MT. Antimicrobial resistance profiles of dairy and clinical isolates and type strains of enterococci. Int J Food Microbiol. 2005;103((2):191–8.View ArticlePubMedGoogle Scholar
Edelstein PH, Meyer RD. Susceptibility of Legionella pneumophila to twenty antimicrobial agents. Antimicrob Agents Chemother. 1980;18(3):403–8.View ArticlePubMedPubMed CentralGoogle Scholar
Finland M, Garner C, Wilcox C, Sabath LD. Susceptibility of recently isolated bacteria to amikacin in vitro: comparisons with four other aminoglycoside antibiotics. J Infect Dis. 1976;134(SUPPL):S297–307.View ArticlePubMedGoogle Scholar
Flamm RK, Sader HS, Jones RN. Ceftaroline activity against organisms isolated from respiratory tract infections in USA hospitals: results from the AWARE Program, 2009–2011. Diagn Microbiol Infect Dis. 2014;78(4):437–42.View ArticlePubMedGoogle Scholar
Forward KR, Low DE, Laverdiere M, Rennie R, Simor AE, Franks PA. Study of the comparative activity of piperacillin/tazobactam with currently available antibiotics against 8206 aerobic isolates. Can J Infect Dis. 1997;8(3):147–53.View ArticlePubMedPubMed CentralGoogle Scholar
Fuchs PC, Barry AL, Thornsberry C, Jones RN. In vitro activity of ticarcillin plus clavulanic acid against 632 clinical isolates. Antimicrob Agents Chemother. 1984;25(3):392–4.View ArticlePubMedPubMed CentralGoogle Scholar
Hamamoto K, Shimizu T, Fujimoto N, Zhang Y, Arai S. In vitro activities of moxifloxacin and other fluoroquinolones against Mycoplasma pneumoniae. Antimicrob Agents Chemother. 2001;45(6):1908–10.View ArticlePubMedPubMed CentralGoogle Scholar
Havlichek D, Saravolatz L, Pohlod D. Effect of quinolones and other antimicrobial agents on cell-associated Legionella pneumophila. Antimicrob Agents Chemother. 1987;31(10):1529–34.View ArticlePubMedPubMed CentralGoogle Scholar
Ikejima H, Yamamoto H, Ishida K, Terakubo S, Kaku M, Shimada J. Comparison of in-vitro activities of SCH27899 and other antibiotics against Mycoplasma pneumoniae. J Infect Chemother. 2001;7(2):121–3.View ArticlePubMedGoogle Scholar
Jones RN, Ballow CH, Biedenbach DJ, Deinhart JA, Schentag JJ. Antimicrobial activity of quinupristin-dalfopristin (RP 59500, Synercid) tested against over 28,000 recent clinical isolates from 200 medical centers in the United States and Canada. Diagn Microbiol Infect Dis. 1998;31(3):437–51.View ArticlePubMedGoogle Scholar
Karlowsky JA, Adam HJ, Decorby MR, Lagacé-wiens PR, Hoban DJ, Zhanel GG. In vitro activity of ceftaroline against gram-positive and gram-negative pathogens isolated from patients in Canadian hospitals in 2009. Antimicrob Agents Chemother. 2011;55(6):2837–46.View ArticlePubMedPubMed CentralGoogle Scholar
Kenny GE, Cartwright FD. Susceptibilities of Mycoplasma hominis, M. pneumoniae, and Ureaplasma urealyticum to GAR-936, dalfopristin, dirithromycin, evernimicin, gatifloxacin, linezolid, moxifloxacin, quinupristin-dalfopristin, and telithromycin compared to their susceptibilities to reference macrolides, tetracyclines, and quinolones. Antimicrob Agents Chemother. 2001;45(9):2604–8.View ArticlePubMedPubMed CentralGoogle Scholar
Kenny GE, Cartwright FD. Susceptibility of Mycoplasma pneumoniae to several new quinolones, tetracycline, and erythromycin. Antimicrob Agents Chemother. 1991;35(3):587–9.View ArticlePubMedPubMed CentralGoogle Scholar
Koeth LM, Good CE, Appelbaum PC, Goldstein EJ, Rodloff AC, Claros M, et al. Surveillance of susceptibility patterns in 1297 European and US anaerobic and capnophilic isolates to co-amoxiclav and five other antimicrobial agents. J Antimicrob Chemother. 2004;53(6):1039–44.View ArticlePubMedGoogle Scholar
Livermore DM, Carter MW, Bagel S, Wiedemann B, Baguero F, Loza E, et al. In vitro activities of ertapenem (MK-0826) against recent clinical bacteria collected in Europe and Australia. Antimicrob Agents Chemother. 2001;45(6):1860–7.View ArticlePubMedPubMed CentralGoogle Scholar
Pasculle AW, Dowling JN, Weyant RS, Sniffen JM, Cordes LG, Gorman GM, et al. Susceptibility of Pittsburgh pneumonia agent (Legionella micdadei) and other newly recognized members of the genus Legionella to nineteen antimicrobial agents. Antimicrob Agents Chemother. 1981;20(6):793–9.View ArticlePubMedPubMed CentralGoogle Scholar
Quentin C, Cantet P, Renaudin H, Bebear C. Antibiotic sensitivity of Mycoplasma pathogenic for man. Pathol Biol. 1985;33(3):205–12.PubMedGoogle Scholar
Rosenblatt JE, Stewart PR. Lack of activity of sulfamethoxazole and trimethoprim against anaerobic bacteria. Antimicrob Agents Chemother. 1974;6(1):93–7.View ArticlePubMedPubMed CentralGoogle Scholar
Sader HS, Farrell DJ, Flamm RK, Jones RN. Variation in potency and spectrum of tigecycline activity against bacterial strains from U.S. medical centers since its approval for clinical use (2006 to 2012). Antimicrob Agents Chemother. 2014;58(4):2274–80.View ArticlePubMedPubMed CentralGoogle Scholar
Schülin T, Wennersten CB, Ferraro MJ, Moellering RC, Eliopoulos GM. Susceptibilities of Legionella spp. to newer antimicrobials in vitro. Antimicrob Agents Chemother. 1998;42(6):1520–3.PubMedPubMed CentralGoogle Scholar
Stout JE, Sens K, Mietzner S, Obman A, Yu VL. Comparative activity of quinolones, macrolides and ketolides against Legionella species using in vitro broth dilution and intracellular susceptibility testing. Int J Antimicrob Agents. 2005;25(4):302–7.View ArticlePubMedGoogle Scholar
Tunér K, Nord CE. Antibiotic susceptibility of anaerobic bacteria in Europe. Clin Infect Dis. 1993;16 Suppl 4:S387–9.View ArticlePubMedGoogle Scholar
Wexler HM. In vitro activity of ertapenem: review of recent studies. J Antimicrob Chemother. 2004;53 Suppl 2:ii11–21.PubMedGoogle Scholar
Zhanel GG, Adam HJ, Baxter MR, Fuller J, Nichol KA, Denisuik AJ, et al. Antimicrobial susceptibility of 22746 pathogens from Canadian hospitals: results of the CANWARD 2007–11 study. J Antimicrob Chemother. 2013;68 Suppl 1:i7–22.View ArticlePubMedGoogle Scholar
Schneider R, Bagby J, Carlson R. Bar-code medication administration: a systems perspective. Am J Health Syst Pharm. 2008;65(23):2216. 8–9.View ArticlePubMedGoogle Scholar
Aronsky D, Haug PJ, Lagor C, Dean NC. Accuracy of administrative data for identifying patients with pneumonia. Am J Med Qual. 2005;20(6):319–28.View ArticlePubMedGoogle Scholar
Jones M, DuVall SL, Spuhl J, Samore MH, Nielson C, Rubin M. Identification of methicillin-resistant Staphylococcus aureus within the nation’s veterans affairs medical centers using natural language processing. BMC Med Inform Decis Mak. 2012;12:34.View ArticlePubMedPubMed CentralGoogle Scholar
Oosterheert JJ, Bonten MJ, Schneider MM, Buskens E, Lammers JW, Hustinx WM, et al. Effectiveness of early switch from intravenous to oral antibiotics in severe community acquired pneumonia: multicentre randomised trial. BMJ. 2006;333(7580):1193. Epub 2006 Nov 7.View ArticlePubMedPubMed CentralGoogle Scholar
Solomkin JS, Reinhart HH, Dellinger EP, Bohnen JM, Rotstein OD, Vogel SB, et al. Results of a randomized trial comparing sequential intravenous/oral treatment with ciprofloxacin plus metronidazole to imipenem/cilastatin for intra-abdominal infections. The Intra-Abdominal Infection Study Group. Ann Surg. 1996;223(3):303–15.View ArticlePubMedPubMed CentralGoogle Scholar
Nussenblatt V, Avdic E, Cosgrove S. What is the role of antimicrobial stewardship in improving outcomes of patients with CAP? Infect Dis Clin North Am. 2013;27(1):211–28.View ArticlePubMedGoogle Scholar
Silva BN, Andriolo RB, Atallah AN, Salomão R. De-escalation of antimicrobial treatment for adults with sepsis, severe sepsis or septic shock. Cochrane Database Syst Rev. 2013;3:CD007934.PubMedGoogle Scholar
Kollef MH, Morrow LE, Niederman MS, Leeper KV, Anzueto A, Benz-Scott L, et al. Clinical characteristics and treatment patterns among patients with ventilator-associated pneumonia. Chest. 2006;129(5):1210–8.View ArticlePubMedGoogle Scholar
Van den Bosch CM, Hulscher ME, Natsch S, Gyssens IC, Prins JM, Geerlings SE, et al. Development of quality indicators for antimicrobial treatment in adults with sepsis. BMC Infect Dis. 2014;14:345.View ArticlePubMedPubMed CentralGoogle Scholar
Buyle FM, Metz-Gercek S, Mechtler R, Kern WV, Robays H, Vogelaers D, et al. Prospective multicentre feasibility study of a quality of care indicator for intravenous to oral switch therapy with highly bioavailable antibiotics. J Antimicrob Chemother. 2012;67((8):2043–6.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
