Misrepresentation of US FDA testing criteria and misunderstanding of the efficacy requirements
David Macinga, GOJO Industries, Inc
9 January 2014
We read with interest the recent publication by Kampf et al. [1] which intends to provide guidance on a critical topic: the influence of application volume on hand coverage and antimicrobial efficacy of alcohol-based hand rubs (ABHR). Kampf et al. compare the in-vivo efficacies of three commercial ABHR products, two of which are tested at an application volume of 1.1 ml, and the third at 2 ml. Although their experimental observations are not unexpected, the authors misrepresent the US Food and Drug Administration (FDA) test methodology and efficacy requirements and apply a double standard in their representation of “manufacturer’s recommended” application volumes. As a result, the authors arrive at unfounded conclusions. We shall, here, take the opportunity to address these issues and clarify what can be legitimately concluded based on data presented in Kampf et al.
First, we must take issue with the Kamp et al. interpretation of “manufacturer’s recommended” application volume. The authors reference a conference abstract by Edmonds et al. [2], which showed that two ABHR based on 70% Ethanol both met the in-vivo efficacy requirements of the FDA when tested at a 1.1-ml application volume, but omitted reference to a recent publication by Macinga et al. [3] that explained the context for the 1.1-ml application volumes. The relevant fact is, the 1.1-ml application volumes were selected for testing by Macinga et al. because these were the measured outputs of the manufacturer’s touch-free dispensers. The stated purpose of the Macinga et al. study was to evaluate the efficacy of the ABHRs at the dispensed volume. Importantly, then, this study provided real-world relevance, as it demonstrated the efficacy of products as used by workers in a healthcare environment. Moreover, it is, to-date, the only study published that provides efficacy data for products tested at dispensed volumes. It may be argued plausibly that the volume delivered from a manufacturer’s dispenser represents their recommended application volume. However, Kampf et al. fail to mention that the measured output of the US touch-free dispenser for the 85% w/w ethanol gel evaluated is 1.0 ml. Instead, they arbitrarily choose to evaluate this product at a volume of 2 ml; that is, an amount equivalent to two actuations of the touch-free dispenser. It is well known that ABHR efficacy is influenced by application volume, so it is not at all surprising that the 85% w/w ethanol product achieved higher log10 reductions than did those tested at 1.1-ml volumes. This data comparison is quite misleading, because it involves an unstated assumption that end users would disinfect hands with two actuations of the 85% ethanol gel dispenser, but only one from the 70% ethanol dispensers. Of note, in a recent study of approximately 27 million dispenser actuations, Hines et al. report that end users actuated a single dose from wall-mounted dispensers 90% of the time, despite an output of only 0.75 ml [4]. Furthermore, we are not aware of any obvious verbiage on the dispensers for the 85% w/w ethanol gel in hospitals that instruct the end user to use two doses from the dispenser. Finally, as demonstrated in Edmonds et al., [5] the 85% w/w ethanol gel at a 2-ml volume failed to meet the FDA’s efficacy criteria when tested specifically according to the methodology required by the FDA. In fact, those results suggested that the user would need to actuate the dispenser at least three times to get an amount of 85% w/w gel sufficient to meeting those criteria. We will discuss later in more detail this discrepancy between the Edmonds et al. data and the data presented in Kampf et al.
Secondly, we contrast the methods used by Kampf et al. with the testing methodology required by the FDA. Specifically, Kampf et al. failed to evaluate the test products according to the required FDA method [6]. Products were, instead, evaluated using the methodologies of ASTM E1174-06 [7] and ASTM E2755-10 [8]. Although each of these methods is more recent in origin than the method published in the FDA regulations, neither has been accepted for providing pivotal data to the FDA, and therefore, data from neither can be used as the basis for claims that products “meet” or “do not meet” FDA efficacy requirements. Furthermore, ASTM E-1174-06 was replaced by a revised version in 2013 [9] and is no longer an active standard. As Kampf et al. acknowledge (in subscript to a table), “methodological differences may explain conflicting results.” It is an absolute truism that the observed efficacy of a product will vary when different test methods are used. This is precisely the reason why it is inappropriate for Kamp et al. to make statements regarding whether products meet FDA efficacy requirements when the relevant FDA method was not used. Of note, in a response to a letter to the editor submitted by a coauthor on the Kampf et al. paper, [10] we provided clarification of the FDA’s test method requirements, which, in this instance, they have chosen to ignore [11]. We have no issue with Kampf using ASTM E2755-10 to test ABHR, as we have previously published showing the advantages of that method versus E1174 [12]. However, efficacy criteria relevant to testing executed according to E2755-10 have yet to be established, so relative performance of products should only be compared using statistics. Kampf et al. do not perform any statistical analyses, so no meaningful conclusions can be drawn regarding differences in product efficacies.
Thirdly, we would like to comment on the conclusions made by Kampf et al. in the context of the actual FDA efficacy requirements. Despite using methods currently not accepted by the FDA, Kampf et al. make conclusions regarding products “meeting FDA efficacy requirements” or not. The FDA requires that products be evaluated not just after a single use, but also after ten consecutive uses, to ensure they remain efficacious under high-frequency, repeated use typical of the healthcare environment. Edmonds et al. used the FDA-specified method to compare the 85% ethanol gel to the 70% ethanol gel and foam at equal application volumes (2 ml) [5]. In that study, the 85% ethanol gel failed to meet the FDA efficacy requirement following Application 10, and efficacy was statistically inferior to that of both the 70% ethanol gel and the 70% ethanol foam. Is it possible that Kampf et al. excluded Application 10 data from their paper due to poor performance of the 85% ethanol product relative to FDA efficacy requirements and that of the 70% ethanol gel and/or foam at Application 10?
Finally, we must point out that the method used by Kampf et al. to evaluate hand coverage is technically flawed. The authors state that each product was supplemented with 1.96% fluorescent dye, and hands were evaluated under UV light to determine degree to which hands were covered. However, this procedure will inherently result in a greater fluorescent signal when a greater volume of product is applied. Furthermore, the authors provide no evidence to show that reduced fluorescent signal on a particular area of the hand correlates with inadequate ABHR coverage. In fact, Macinga et al. demonstrated that a 1.1-ml application volume was sufficient to keeping hands wet for more than 20 seconds, which is within the range of 20s-30s specified in the WHO guidelines [3, 13]. Furthermore, and more importantly, the 1.1-ml application volume was sufficient to covering the hands such that the FDA efficacy requirements were met following both the 1st and the 10th application.
In summary, there are multiple issues of concern in the study conducted by Kampf et al, and the conclusions are unjustified and misleading to the reader. It is disingenuous to designate the dispensed volume of two test products the “recommended dose,” but portray a volume equivalent to twice the dispensed volume to be the “recommended dose” for the third, particularly when dispensers of the latter provide no guidance for the end user that two doses should be used. To compare efficacy of products fairly, they should be tested in a single study either at equivalent volumes (as was done in Edmonds et al.), or at the volumes actually dispensed from the manufacturer’s dispensers. And, it is entirely inappropriate to claim that product efficacy does or does not satisfy FDA efficacy criteria when the appropriate test method was not used, and products were not tested for the required number of applications. At best, all that can be concluded from Kampf et al is that a 2-ml volume of one ABHR produced greater log10 reductions than did 1.1-ml volumes of two other ABHR, statistical significance unknown. As other researchers have previously demonstrated that applied volume is a critical determinant of antimicrobial efficacy, this is certainly no novel finding. Unfortunately, Kampf et al. have attempted to clarify recommendations for effective ABHR application volumes, but they have only contributed confusion.
Sincerely, David R. Macinga, PhD, Research Fellow, Microbiology and Clinical Science and Sarah L. Edmonds, MS, Senior Scientist, Clinical Science
References
Kampf G, Ruselack S, Eggerstedt S, Nowak N, Bashir M: Less and less-influence of volume on hand coverage and bactericidal efficacy in hand disinfection.BMC Infect Dis 2013, 13:472.
Edmonds S, Macinga DR, Paulson D: The influence of ABHR product format on in vivo efficacy: a meta-analysis.Am J Infect Contr 2012, 40:e43.
Macinga, D., Edmonds, S., Campbell, E., Shumaker, D., Arbogast, J.: Efficacy of Novel Alcohol-Based Hand Rubs at Typical “In Use” Volumes.Infect Contr Hosp Epidemiol, 2013 34:299-301.
Hines J, Alper P, Voss A, McGeer A: P101: Product dose considerations for real-world hand sanitiser efficacy.Antimicrob Res Infect Contr 2013, 2(Suppl 1):P101.
Edmonds SL, Macinga DR, Mays-Suko P, Duley C, Rutter J, Jarvis WR, Arbogast JW: Comparative Efficacy of Commercially Available Alcohol-Based Hand Rubs and WHO-Recommended Hand Rubs: Formulation Matters.Am J Infect Control 2012, 40:521-525.
Food and Drug Administration. Tentative final monograph for healthcare antiseptic drug products; proposed rule. Federal Register 1994 59:31441-31450.
ASTM International. E-1174-06: Standard Test Method for Evaluation of the Effectiveness of Health Care Personnel or Consumer Handwash Formulations. West Conshohocken, PA: ASTM International, 2006
ASTM International. E-2275-10: Standard Test Method for Determining the Bacteria-Eliminating Effectiveness of Hand Sanitizer Formulations Using Hands of Adults. West Conshohocken, PA: ASTM International, 2010.
ASTM International. E-1174-13: Standard Test Method for Evaluation of the Effectiveness of Health Care Personnel or Consumer Handwash Formulations. West Conshohocken, PA: ASTM International, 2013.
10. Eggerstedt S: Letter to the editor on "Comparative efficacy of commercially available alcohol-based hand rubs and World Health Organization-recommended hand rubs". 2013. Am J Infect Contr41:472-4.
11. Edmonds SL, Macinga DR, Jarvis WR: Reply to letter to the editor on "Comparative efficacy of commercially available alcohol-based hand rubs and World Health Organization-recommended hand rubs". 2013. Am J Infect Contr 41:474-5.
12. Macinga DR, Beausoleil CM, Campbell E, Mulberry G, Brady A, Edmonds SL, Arbogast JW: Quest for a realistic in vivo test method for antimicrobial hand-rub agents: introduction of a low-volume hand contamination procedure.Appl Environ Microbiol 2011, 77:8588-94.
13. World Health Organization. Who guidelines on hand hygiene in health care. Geneva, Switzerland: World Health Organization, 2009.
Competing interests
We are both employees of GOJO Industries, a manufaturer of skin care products including alcohol based hand rubs. We both work in Research and Development.
It is a matter of credibility - Reply to the comments by Macinga and Edmonds
Guenter Kampf, BODE SCIENCE CENTER, Bode Chemie GmbH, Hamburg, Germany
30 January 2014
We would like to respond to the comments by Macinga and Edmonds [1], whose main conclusion appears to be that our recent study concerning alcohol-based hand rub volumes [2] only contributed confusion.
Our study showed quite clearly that hand rub volumes of less than 2 mL were associated with a substantially higher proportion of incompletely covered hands than larger volumes, regardless of the type and brand of hand rub tested [2]. Furthermore, we found a significant correlation between the applied volume and the frequency of incompletely covered hands [2]. These findings offer clarification concerning the suitability of small application volumes such as 1.1 mL that had been recommended by manufacturers of hand rubs. These data indicate that manufacturers who support the use of such small volumes, and health care workers who may trust that such small volumes are sufficient and effective, should reevaluate their recommendations and their practice to ensure that patient safety is not compromised.
Soiling in accordance with the intended use of hand rubs
We evaluated the antimicrobial efficacy of all three hand rubs for only one application with minimal hand soiling, in accordance with their intended use [2]. The intended use of alcohol-based hand rubs, as specified by the US Centers for Disease Control and Prevention (CDC) and the World Health Organization (WHO) hand hygiene guidelines [3, 4], is for hands that are not visibly soiled or dirty. The requirements of the United States Food and Drug Administration (FDA) are based on the tentative final monograph for health care antiseptic products [5]. The FDA-specified test method has been designed for antimicrobial health care personnel hand wash agents (rinse-off formulations) and is in its basic design the same as the American Society for Testing and Materials (ASTM) test method E1174 [6]. ASTM E 1174 requires hand contamination with a 4.5 mL broth-based contamination fluid (according to the FDA it is even 5 mL), followed by the hand wash under investigation and the sampling using a specific sampling solution. This procedure is performed 10 times. The repetitive contamination and sampling increases the amount of soiling of both hands, which is not a problem for hand wash testing. However, with alcohol-based hand antisepsis (leave-on formulations), an increasing amount of organic matter will accumulate on the hands, and they will be substantially soiled after only a few applications [7]. Macinga and Edmonds acknowledge in an earlier article that the soiling on hands is considerably higher when using the ASTM E1174 test method (contamination fluid, 4.5 mL) than with ASTM E2755 (contamination fluid, 0.2 mL) [8].
As a consequence of all the above, is not meaningful to test the efficacy of hands rubs on grossly soiled hands, and this would be inconsistent with clinical practice [7]. Therefore, we evaluated the efficacy of the three hand rubs for only one application with minimum soiling, as previously reported [9]. We also note that the clinical relevance for alcohol-based hand rubs of the ASTM’s testing requirement after the tenth treatment has been questioned by others [10]. Thus, it is clear that our experiments captured both the primary intent of the FDA requirements as well as the clinically relevant aspects of hand contamination.
Adequate volumes of alcohol-based hand rub
Macinga and Edmonds question the manufacturer’s recommended application volume of 2 mL for the 85% ethanol gel and the justification for this volume [1]. In a study [11] that was published after our article [2] was submitted in February 2013, a volume of 1.1 mL of a gel and a foam (the other two products tested in our study) was tested, and the stated reason for using this volume was that this corresponded to the measured output from the manufacturer’s touch-free dispenser. We certainly note that the volume of 1.1 mL has become the subject of a manufacturer’s advertisement [12]. On the other hand, although it may be an advantage for the end user if the dispensed volume per single dispenser action is identical with the manufacturer’s recommended volume, it is simply not possible to infer this for all available hand rubs and all types of dispensers. There are various types of dispensers in use in different countries and health care settings, including wall-mounted lever-driven, wall-mounted touch-free or mobile dosing-pump dispensers, and each may have different dispensed volumes. The assumption by Macinga and Edmonds that the dispensed volume of 1 mL from one dispenser ‘by default’ represents a manufacturer’s recommended volume is simply incorrect, and this is inconsistent with a real-world scenario. Unless explicitly stated – which, however, is fairly obvious when there is an advertisement to that effect – the dispensed volume (dispenser) cannot be assumed to be identical with a manufacturer’s recommended volume (hand rub).
It is necessary to show (a) that the recommended volume of a hand rub fulfills antimicrobial efficacy standards and (b) that it is sufficient to cover both hands. In a second stage, health care workers need to be educated about the volume necessary for efficacy for each application. The tested volume of 2 mL for the 85% ethanol gel product was not chosen arbitrarily as stated by Macinga and Edmonds; this volume was chosen based on previously obtained efficacy data. Although the measured output of the touch-free dispenser for the 85% ethanol gel in the United States is 1 mL, a minimum volume of 2 mL can be recommended for this product regardless of the type of dispenser used. A 1-mL dispenser allows an accurate dose of 2 mL when 2 aliquots are dispensed.
Current efficacy testing standards and FDA criteria
Macinga and Edmonds question the suitability of the methods that we chose (ASTM E1174-06 and ASTM E2755-10) to evaluate whether hand rubs meet the efficacy requirements of the FDA, stating that only the FDA’s required method [5] can lead to such an assessment. ASTM E 1174, with minor modifications, has apparently been accepted by the US Food and Drug Administration as a validated method for submitting a label claim as an antiseptic [13]. Macinga and Edmonds published a study that used the “ASTM E1174-96” method for alcohol-based hand rubs. To our knowledge this method version does formally not even exist. They probably applied “ASTM E1174-94” and concluded that two of the tested products satisfied the health care professional hand wash requirements of the FDA [14]. Although that study was published in 2012, revisions of ASTM E1174-94 had been published in 2000 and 2006 (“historical versions of ASTM E 1174”; www.astm.org). If one concludes that ASTM E1174-06 should not be used to judge whether a product meets FDA efficacy criteria because the method is not identical with the FDA method, then one should equally conclude that ASTM E1174-94 should not be used. In another article [11] that meanwhile constitutes the key reference by Macinga and Edmonds that describes the testing of the 1.1 mL hand rub volumes, the FDA method is cited as ASTM E1174-06 (not -94) in the methods section, and conclusions concerning FDA efficacy requirements are definitely made. Similarly, in another article [8], the authors designed a new test method that subsequently became the ASTM E2755 standard [15] and proposed that it should be adopted for the routine evaluation of alcohol-based hand rubs for regulatory purposes [8]. Thus, we see obvious inconsistencies within the same line of arguments between Macinga and Edmonds’ previously published articles and their present comment [1].
The authors also question our use of the ASTM E1174-06 standard, stating that this is no longer an active standard because it has been superseded by ASTM E1174-13, which was published in 2013. The simple fact is that our experiments were performed in 2012, using the available up-to-date standard at the time [2]. It is clear that both ASTM methods (E1174 and E2755) include a description of the test methods, but no pass or fail criteria. This justifies the suitability of these methods to evaluate efficacy. The FDA requirements for antimicrobial soaps (health care personnel hand wash) were defined in 1994 [5], and we applied the same efficacy criteria in our study [2].
Hand surface coverage by alcohol-based hand rubs
Macinga and Edmonds further question the suitability of our method to determine hand coverage using fluorescent dye and an ultraviolet lamp, describing it as “technically flawed” and stating that that the overall amount of fluorescent signal will be greater when a greater amount of dye-supplemented hand rub will be applied. They, however, miss the point of our experiments. As described quite clearly in our article [2], hands were assessed for gaps in fluorescence (i.e. whether there were areas with no fluorescence at all), because such gaps provide the best indication of incomplete hand surface coverage. The overall intensity of fluorescent signal was not assessed. In terms of the queried evidence, a clear gap (i.e. a completely untreated area) arguably constitutes strong evidence of incomplete coverage.
In a previous study by the authors [11], the drying time of 1.1 mL of two different hand rubs was slightly more than 20 s, and we found very similar drying times of 20-25 s for three hand rubs (1.1 mL) in our study [2]. Macinga and Edmonds further claim that 1.1 mL is sufficient in terms of hand surface coverage, based on a test method to determine antimicrobial efficacy [11]. However, the cited study [11] contains no description of a test method to assess hand surface coverage, and coverage data are not provided. It is clear that the use of a standard antimicrobial efficacy method does not allow statements to be made about hand surface coverage. This must be evaluated with a specific method, such as the fluorescence gap method that we described [2].
Finally, we think – and this is certainly up for separate judgment by the journal’s readers – that the wording in some parts of the comment by Macinga and Edmonds is beyond what we consider appropriate in a scientific discourse.
Günter Kampf,1, 2 Sigunde Ruselack,3 Sven Eggerstedt,3 Nicolas Nowak4, Muhammad Bashir5
5MicroBioTest Division of Microbac Laboratories, Inc., 105 Carpenter Drive, Sterling, Virginia, USA
REFERENCES
1. Macinga DR, Edmonds SL: Misinterpretation of the US FDA testing criteria and misunderstanding of the efficacy requirements (comment). BMC Infect Dis 2014, 14:comment.
2. Kampf G, Ruselack S, Eggerstedt S, Nowak N, Bashir M: Less and less-influence of volume on hand coverage and bactericidal efficacy in hand disinfection. BMC Infect Dis 2013, 13:472.
3. Boyce JM, Pittet D: Guideline for hand hygiene in health-care settings. Recommendations of the healthcare infection control practices advisory committee and the HICPAC/SHEA/APIC/IDSA hand hygiene task force. MMWR - Morbidity & Mortality Weekly Report 2002, 51:1-45.
4. Anonymous: WHO guidelines on hand hygiene in health care. First Global Patient Safety Challenge Clean Care is Safer Care. Geneva: WHO; 2009.
5. Anonymous: Tentative final monograph for health care antiseptic products; proposed rule. Federal Register 1994, 59(116):31401-31452.
6. American Society for Testing and Materials International: ASTM E 1174 - 06: Standard test method for evaluation of the effectiveness of healthcare personnel handwash formulations. In. West Conshohocken [PA]: American Society for Testing and Materials; 2006.
7. Kampf G, Kramer A: Efficacy of hand hygiene agents at short application times. American Journal of Infection Control 2005, 33(7):429-431.
8. Macinga DR, Beausoleil CM, Campbell E, Mulberry G, Brady A, Edmonds SL, Arbogast JW: Quest for a realistic in vivo test method for antimicrobial hand-rub agents: introduction of a low-volume hand contamination procedure. Appl Environ Microbiol 2011, 77(24):8588-8594.
9. Kampf G: How effective are hand antiseptics for the post-contamination treatment of hands when used as recommended? American Journal of Infection Control 2008, 36(5):356-360.
10. Rotter M, Sattar S, Dharan S, Allegranzi B, Mathai E, Pittet D: Methods to evaluate the microbicidal activities of hand-rub and hand-wash agents. J Hosp Infect 2009, 73(3):191-199.
11. Macinga DR, Edmonds SL, Campbell E, Shumaker DJ, Arbogast JW: Efficacy of novel alcohol-based hand rub products at typical in-use volumes. Infect Control Hosp Epidemiol 2013, 34(3):299-301.
12. Anonymous: The best way to kill germs in one pump (advertisement by GOJO Industries). Infection Control Today 2013.
13. Weber DJ, Sickbert-Bennett E, Gergen MF, Rutala WA: Efficacy of selected hand hygiene agents used to remove Bacillus atrophaeus (a surrogate of Bacillus anthracis) from contaminated hands. The Journal of the American Medical Association 2003, 289(10):1274-1277.
14. Edmonds SL, Macinga DR, Mays-Suko P, Duley C, Rutter J, Jarvis WR, Arbogast JW: Comparative efficacy of commercially available alcohol-based hand rubs and World Health Organization-recommended hand rubs: formulation matters. American Journal of Infection Control 2012, 40(6):521-525.
15. American Society for Testing and Materials International: ASTM E 2755 - 10, Standard test method for determining the bacteria-eliminating effectiveness of hand sanitizer formulations using hands of adults. In. West Conshohocken, PA: American Society for Testing and Materials International,; 2010.
Competing interests
The first four authors are paid employees of Bode Chemie GmbH, Hamburg, Germany.
Misrepresentation of US FDA testing criteria and misunderstanding of the efficacy requirements
9 January 2014
We read with interest the recent publication by Kampf et al. [1] which intends to provide guidance on a critical topic: the influence of application volume on hand coverage and antimicrobial efficacy of alcohol-based hand rubs (ABHR). Kampf et al. compare the in-vivo efficacies of three commercial ABHR products, two of which are tested at an application volume of 1.1 ml, and the third at 2 ml. Although their experimental observations are not unexpected, the authors misrepresent the US Food and Drug Administration (FDA) test methodology and efficacy requirements and apply a double standard in their representation of “manufacturer’s recommended” application volumes. As a result, the authors arrive at unfounded conclusions. We shall, here, take the opportunity to address these issues and clarify what can be legitimately concluded based on data presented in Kampf et al.
First, we must take issue with the Kamp et al. interpretation of “manufacturer’s recommended” application volume. The authors reference a conference abstract by Edmonds et al. [2], which showed that two ABHR based on 70% Ethanol both met the in-vivo efficacy requirements of the FDA when tested at a 1.1-ml application volume, but omitted reference to a recent publication by Macinga et al. [3] that explained the context for the 1.1-ml application volumes. The relevant fact is, the 1.1-ml application volumes were selected for testing by Macinga et al. because these were the measured outputs of the manufacturer’s touch-free dispensers. The stated purpose of the Macinga et al. study was to evaluate the efficacy of the ABHRs at the dispensed volume. Importantly, then, this study provided real-world relevance, as it demonstrated the efficacy of products as used by workers in a healthcare environment. Moreover, it is, to-date, the only study published that provides efficacy data for products tested at dispensed volumes. It may be argued plausibly that the volume delivered from a manufacturer’s dispenser represents their recommended application volume. However, Kampf et al. fail to mention that the measured output of the US touch-free dispenser for the 85% w/w ethanol gel evaluated is 1.0 ml. Instead, they arbitrarily choose to evaluate this product at a volume of 2 ml; that is, an amount equivalent to two actuations of the touch-free dispenser. It is well known that ABHR efficacy is influenced by application volume, so it is not at all surprising that the 85% w/w ethanol product achieved higher log10 reductions than did those tested at 1.1-ml volumes. This data comparison is quite misleading, because it involves an unstated assumption that end users would disinfect hands with two actuations of the 85% ethanol gel dispenser, but only one from the 70% ethanol dispensers. Of note, in a recent study of approximately 27 million dispenser actuations, Hines et al. report that end users actuated a single dose from wall-mounted dispensers 90% of the time, despite an output of only 0.75 ml [4]. Furthermore, we are not aware of any obvious verbiage on the dispensers for the 85% w/w ethanol gel in hospitals that instruct the end user to use two doses from the dispenser. Finally, as demonstrated in Edmonds et al., [5] the 85% w/w ethanol gel at a 2-ml volume failed to meet the FDA’s efficacy criteria when tested specifically according to the methodology required by the FDA. In fact, those results suggested that the user would need to actuate the dispenser at least three times to get an amount of 85% w/w gel sufficient to meeting those criteria. We will discuss later in more detail this discrepancy between the Edmonds et al. data and the data presented in Kampf et al.
Secondly, we contrast the methods used by Kampf et al. with the testing methodology required by the FDA. Specifically, Kampf et al. failed to evaluate the test products according to the required FDA method [6]. Products were, instead, evaluated using the methodologies of ASTM E1174-06 [7] and ASTM E2755-10 [8]. Although each of these methods is more recent in origin than the method published in the FDA regulations, neither has been accepted for providing pivotal data to the FDA, and therefore, data from neither can be used as the basis for claims that products “meet” or “do not meet” FDA efficacy requirements. Furthermore, ASTM E-1174-06 was replaced by a revised version in 2013 [9] and is no longer an active standard. As Kampf et al. acknowledge (in subscript to a table), “methodological differences may explain conflicting results.” It is an absolute truism that the observed efficacy of a product will vary when different test methods are used. This is precisely the reason why it is inappropriate for Kamp et al. to make statements regarding whether products meet FDA efficacy requirements when the relevant FDA method was not used. Of note, in a response to a letter to the editor submitted by a coauthor on the Kampf et al. paper, [10] we provided clarification of the FDA’s test method requirements, which, in this instance, they have chosen to ignore [11]. We have no issue with Kampf using ASTM E2755-10 to test ABHR, as we have previously published showing the advantages of that method versus E1174 [12]. However, efficacy criteria relevant to testing executed according to E2755-10 have yet to be established, so relative performance of products should only be compared using statistics. Kampf et al. do not perform any statistical analyses, so no meaningful conclusions can be drawn regarding differences in product efficacies.
Thirdly, we would like to comment on the conclusions made by Kampf et al. in the context of the actual FDA efficacy requirements. Despite using methods currently not accepted by the FDA, Kampf et al. make conclusions regarding products “meeting FDA efficacy requirements” or not. The FDA requires that products be evaluated not just after a single use, but also after ten consecutive uses, to ensure they remain efficacious under high-frequency, repeated use typical of the healthcare environment. Edmonds et al. used the FDA-specified method to compare the 85% ethanol gel to the 70% ethanol gel and foam at equal application volumes (2 ml) [5]. In that study, the 85% ethanol gel failed to meet the FDA efficacy requirement following Application 10, and efficacy was statistically inferior to that of both the 70% ethanol gel and the 70% ethanol foam. Is it possible that Kampf et al. excluded Application 10 data from their paper due to poor performance of the 85% ethanol product relative to FDA efficacy requirements and that of the 70% ethanol gel and/or foam at Application 10?
Finally, we must point out that the method used by Kampf et al. to evaluate hand coverage is technically flawed. The authors state that each product was supplemented with 1.96% fluorescent dye, and hands were evaluated under UV light to determine degree to which hands were covered. However, this procedure will inherently result in a greater fluorescent signal when a greater volume of product is applied. Furthermore, the authors provide no evidence to show that reduced fluorescent signal on a particular area of the hand correlates with inadequate ABHR coverage. In fact, Macinga et al. demonstrated that a 1.1-ml application volume was sufficient to keeping hands wet for more than 20 seconds, which is within the range of 20s-30s specified in the WHO guidelines [3, 13]. Furthermore, and more importantly, the 1.1-ml application volume was sufficient to covering the hands such that the FDA efficacy requirements were met following both the 1st and the 10th application.
In summary, there are multiple issues of concern in the study conducted by Kampf et al, and the conclusions are unjustified and misleading to the reader. It is disingenuous to designate the dispensed volume of two test products the “recommended dose,” but portray a volume equivalent to twice the dispensed volume to be the “recommended dose” for the third, particularly when dispensers of the latter provide no guidance for the end user that two doses should be used. To compare efficacy of products fairly, they should be tested in a single study either at equivalent volumes (as was done in Edmonds et al.), or at the volumes actually dispensed from the manufacturer’s dispensers. And, it is entirely inappropriate to claim that product efficacy does or does not satisfy FDA efficacy criteria when the appropriate test method was not used, and products were not tested for the required number of applications. At best, all that can be concluded from Kampf et al is that a 2-ml volume of one ABHR produced greater log10 reductions than did 1.1-ml volumes of two other ABHR, statistical significance unknown. As other researchers have previously demonstrated that applied volume is a critical determinant of antimicrobial efficacy, this is certainly no novel finding. Unfortunately, Kampf et al. have attempted to clarify recommendations for effective ABHR application volumes, but they have only contributed confusion.
Sincerely, David R. Macinga, PhD, Research Fellow, Microbiology and Clinical Science and Sarah L. Edmonds, MS, Senior Scientist, Clinical Science
References
Competing interests
We are both employees of GOJO Industries, a manufaturer of skin care products including alcohol based hand rubs. We both work in Research and Development.It is a matter of credibility - Reply to the comments by Macinga and Edmonds
30 January 2014
We would like to respond to the comments by Macinga and Edmonds [1], whose main conclusion appears to be that our recent study concerning alcohol-based hand rub volumes [2] only contributed confusion.
Our study showed quite clearly that hand rub volumes of less than 2 mL were associated with a substantially higher proportion of incompletely covered hands than larger volumes, regardless of the type and brand of hand rub tested [2]. Furthermore, we found a significant correlation between the applied volume and the frequency of incompletely covered hands [2]. These findings offer clarification concerning the suitability of small application volumes such as 1.1 mL that had been recommended by manufacturers of hand rubs. These data indicate that manufacturers who support the use of such small volumes, and health care workers who may trust that such small volumes are sufficient and effective, should reevaluate their recommendations and their practice to ensure that patient safety is not compromised.
Soiling in accordance with the intended use of hand rubs
We evaluated the antimicrobial efficacy of all three hand rubs for only one application with minimal hand soiling, in accordance with their intended use [2]. The intended use of alcohol-based hand rubs, as specified by the US Centers for Disease Control and Prevention (CDC) and the World Health Organization (WHO) hand hygiene guidelines [3, 4], is for hands that are not visibly soiled or dirty. The requirements of the United States Food and Drug Administration (FDA) are based on the tentative final monograph for health care antiseptic products [5]. The FDA-specified test method has been designed for antimicrobial health care personnel hand wash agents (rinse-off formulations) and is in its basic design the same as the American Society for Testing and Materials (ASTM) test method E1174 [6]. ASTM E 1174 requires hand contamination with a 4.5 mL broth-based contamination fluid (according to the FDA it is even 5 mL), followed by the hand wash under investigation and the sampling using a specific sampling solution. This procedure is performed 10 times. The repetitive contamination and sampling increases the amount of soiling of both hands, which is not a problem for hand wash testing. However, with alcohol-based hand antisepsis (leave-on formulations), an increasing amount of organic matter will accumulate on the hands, and they will be substantially soiled after only a few applications [7]. Macinga and Edmonds acknowledge in an earlier article that the soiling on hands is considerably higher when using the ASTM E1174 test method (contamination fluid, 4.5 mL) than with ASTM E2755 (contamination fluid, 0.2 mL) [8].
As a consequence of all the above, is not meaningful to test the efficacy of hands rubs on grossly soiled hands, and this would be inconsistent with clinical practice [7]. Therefore, we evaluated the efficacy of the three hand rubs for only one application with minimum soiling, as previously reported [9]. We also note that the clinical relevance for alcohol-based hand rubs of the ASTM’s testing requirement after the tenth treatment has been questioned by others [10]. Thus, it is clear that our experiments captured both the primary intent of the FDA requirements as well as the clinically relevant aspects of hand contamination.
Adequate volumes of alcohol-based hand rub
Macinga and Edmonds question the manufacturer’s recommended application volume of 2 mL for the 85% ethanol gel and the justification for this volume [1]. In a study [11] that was published after our article [2] was submitted in February 2013, a volume of 1.1 mL of a gel and a foam (the other two products tested in our study) was tested, and the stated reason for using this volume was that this corresponded to the measured output from the manufacturer’s touch-free dispenser. We certainly note that the volume of 1.1 mL has become the subject of a manufacturer’s advertisement [12]. On the other hand, although it may be an advantage for the end user if the dispensed volume per single dispenser action is identical with the manufacturer’s recommended volume, it is simply not possible to infer this for all available hand rubs and all types of dispensers. There are various types of dispensers in use in different countries and health care settings, including wall-mounted lever-driven, wall-mounted touch-free or mobile dosing-pump dispensers, and each may have different dispensed volumes. The assumption by Macinga and Edmonds that the dispensed volume of 1 mL from one dispenser ‘by default’ represents a manufacturer’s recommended volume is simply incorrect, and this is inconsistent with a real-world scenario. Unless explicitly stated – which, however, is fairly obvious when there is an advertisement to that effect – the dispensed volume (dispenser) cannot be assumed to be identical with a manufacturer’s recommended volume (hand rub).
It is necessary to show (a) that the recommended volume of a hand rub fulfills antimicrobial efficacy standards and (b) that it is sufficient to cover both hands. In a second stage, health care workers need to be educated about the volume necessary for efficacy for each application. The tested volume of 2 mL for the 85% ethanol gel product was not chosen arbitrarily as stated by Macinga and Edmonds; this volume was chosen based on previously obtained efficacy data. Although the measured output of the touch-free dispenser for the 85% ethanol gel in the United States is 1 mL, a minimum volume of 2 mL can be recommended for this product regardless of the type of dispenser used. A 1-mL dispenser allows an accurate dose of 2 mL when 2 aliquots are dispensed.
Current efficacy testing standards and FDA criteria
Macinga and Edmonds question the suitability of the methods that we chose (ASTM E1174-06 and ASTM E2755-10) to evaluate whether hand rubs meet the efficacy requirements of the FDA, stating that only the FDA’s required method [5] can lead to such an assessment. ASTM E 1174, with minor modifications, has apparently been accepted by the US Food and Drug Administration as a validated method for submitting a label claim as an antiseptic [13]. Macinga and Edmonds published a study that used the “ASTM E1174-96” method for alcohol-based hand rubs. To our knowledge this method version does formally not even exist. They probably applied “ASTM E1174-94” and concluded that two of the tested products satisfied the health care professional hand wash requirements of the FDA [14]. Although that study was published in 2012, revisions of ASTM E1174-94 had been published in 2000 and 2006 (“historical versions of ASTM E 1174”; www.astm.org). If one concludes that ASTM E1174-06 should not be used to judge whether a product meets FDA efficacy criteria because the method is not identical with the FDA method, then one should equally conclude that ASTM E1174-94 should not be used. In another article [11] that meanwhile constitutes the key reference by Macinga and Edmonds that describes the testing of the 1.1 mL hand rub volumes, the FDA method is cited as ASTM E1174-06 (not -94) in the methods section, and conclusions concerning FDA efficacy requirements are definitely made. Similarly, in another article [8], the authors designed a new test method that subsequently became the ASTM E2755 standard [15] and proposed that it should be adopted for the routine evaluation of alcohol-based hand rubs for regulatory purposes [8]. Thus, we see obvious inconsistencies within the same line of arguments between Macinga and Edmonds’ previously published articles and their present comment [1].
The authors also question our use of the ASTM E1174-06 standard, stating that this is no longer an active standard because it has been superseded by ASTM E1174-13, which was published in 2013. The simple fact is that our experiments were performed in 2012, using the available up-to-date standard at the time [2]. It is clear that both ASTM methods (E1174 and E2755) include a description of the test methods, but no pass or fail criteria. This justifies the suitability of these methods to evaluate efficacy. The FDA requirements for antimicrobial soaps (health care personnel hand wash) were defined in 1994 [5], and we applied the same efficacy criteria in our study [2].
Hand surface coverage by alcohol-based hand rubs
Macinga and Edmonds further question the suitability of our method to determine hand coverage using fluorescent dye and an ultraviolet lamp, describing it as “technically flawed” and stating that that the overall amount of fluorescent signal will be greater when a greater amount of dye-supplemented hand rub will be applied. They, however, miss the point of our experiments. As described quite clearly in our article [2], hands were assessed for gaps in fluorescence (i.e. whether there were areas with no fluorescence at all), because such gaps provide the best indication of incomplete hand surface coverage. The overall intensity of fluorescent signal was not assessed. In terms of the queried evidence, a clear gap (i.e. a completely untreated area) arguably constitutes strong evidence of incomplete coverage.
In a previous study by the authors [11], the drying time of 1.1 mL of two different hand rubs was slightly more than 20 s, and we found very similar drying times of 20-25 s for three hand rubs (1.1 mL) in our study [2]. Macinga and Edmonds further claim that 1.1 mL is sufficient in terms of hand surface coverage, based on a test method to determine antimicrobial efficacy [11]. However, the cited study [11] contains no description of a test method to assess hand surface coverage, and coverage data are not provided. It is clear that the use of a standard antimicrobial efficacy method does not allow statements to be made about hand surface coverage. This must be evaluated with a specific method, such as the fluorescence gap method that we described [2].
Finally, we think – and this is certainly up for separate judgment by the journal’s readers – that the wording in some parts of the comment by Macinga and Edmonds is beyond what we consider appropriate in a scientific discourse.
Günter Kampf,1, 2 Sigunde Ruselack,3 Sven Eggerstedt,3 Nicolas Nowak4, Muhammad Bashir5
1BODE SCIENCE CENTER, Bode Chemie GmbH, Melanchthonstrasse 27, 22525 Hamburg, Germany
2Institut für Hygiene und Umweltmedizin, Ernst-Moritz-Arndt Universität Greifswald, Walther-Rathenau-Strasse 49a, 17475 Greifswald, Germany
3Development, Bode Chemie GmbH, Melanchthonstrasse 27, 22525 Hamburg, Germany
4Scientific Affairs, Bode Chemie GmbH, Melanchthonstrasse 27, 22525 Hamburg, Germany
5MicroBioTest Division of Microbac Laboratories, Inc., 105 Carpenter Drive, Sterling, Virginia, USA
REFERENCES
1. Macinga DR, Edmonds SL: Misinterpretation of the US FDA testing criteria and misunderstanding of the efficacy requirements (comment). BMC Infect Dis 2014, 14:comment.
2. Kampf G, Ruselack S, Eggerstedt S, Nowak N, Bashir M: Less and less-influence of volume on hand coverage and bactericidal efficacy in hand disinfection. BMC Infect Dis 2013, 13:472.
3. Boyce JM, Pittet D: Guideline for hand hygiene in health-care settings. Recommendations of the healthcare infection control practices advisory committee and the HICPAC/SHEA/APIC/IDSA hand hygiene task force. MMWR - Morbidity & Mortality Weekly Report 2002, 51:1-45.
4. Anonymous: WHO guidelines on hand hygiene in health care. First Global Patient Safety Challenge Clean Care is Safer Care. Geneva: WHO; 2009.
5. Anonymous: Tentative final monograph for health care antiseptic products; proposed rule. Federal Register 1994, 59(116):31401-31452.
6. American Society for Testing and Materials International: ASTM E 1174 - 06: Standard test method for evaluation of the effectiveness of healthcare personnel handwash formulations. In. West Conshohocken [PA]: American Society for Testing and Materials; 2006.
7. Kampf G, Kramer A: Efficacy of hand hygiene agents at short application times. American
Journal of Infection Control 2005, 33(7):429-431.
8. Macinga DR, Beausoleil CM, Campbell E, Mulberry G, Brady A, Edmonds SL, Arbogast JW: Quest for a realistic in vivo test method for antimicrobial hand-rub agents: introduction of a low-volume hand contamination procedure. Appl Environ Microbiol 2011, 77(24):8588-8594.
9. Kampf G: How effective are hand antiseptics for the post-contamination treatment of hands when used as recommended? American Journal of Infection Control 2008, 36(5):356-360.
10. Rotter M, Sattar S, Dharan S, Allegranzi B, Mathai E, Pittet D: Methods to evaluate the microbicidal activities of hand-rub and hand-wash agents. J Hosp Infect 2009, 73(3):191-199.
11. Macinga DR, Edmonds SL, Campbell E, Shumaker DJ, Arbogast JW: Efficacy of novel alcohol-based hand rub products at typical in-use volumes. Infect Control Hosp Epidemiol 2013, 34(3):299-301.
12. Anonymous: The best way to kill germs in one pump (advertisement by GOJO Industries). Infection Control Today 2013.
13. Weber DJ, Sickbert-Bennett E, Gergen MF, Rutala WA: Efficacy of selected hand hygiene agents used to remove Bacillus atrophaeus (a surrogate of Bacillus anthracis) from contaminated hands. The Journal of the American Medical Association 2003, 289(10):1274-1277.
14. Edmonds SL, Macinga DR, Mays-Suko P, Duley C, Rutter J, Jarvis WR, Arbogast JW: Comparative efficacy of commercially available alcohol-based hand rubs and World Health Organization-recommended hand rubs: formulation matters. American Journal of Infection Control 2012, 40(6):521-525.
15. American Society for Testing and Materials International: ASTM E 2755 - 10, Standard test method for determining the bacteria-eliminating effectiveness of hand sanitizer formulations using hands of adults. In. West Conshohocken, PA: American Society for Testing and Materials International,; 2010.
Competing interests
The first four authors are paid employees of Bode Chemie GmbH, Hamburg, Germany.