Misrepresentation of US FDA testing criteria and misunderstanding of the efficacy requirements David Macinga, GOJO Industries, Inc 9 January 2014 We read with interest the recent publication by Kampf et al.  which intends to provide guidance on a critical topic: the influence of application volume on hand coverage and antimicrobial efficacy of alcohol-based hand rubs (ABHR). Kampf et al. compare the in-vivo efficacies of three commercial ABHR products, two of which are tested at an application volume of 1.1 ml, and the third at 2 ml. Although their experimental observations are not unexpected, the authors misrepresent the US Food and Drug Administration (FDA) test methodology and efficacy requirements and apply a double standard in their representation of “manufacturer’s recommended” application volumes. As a result, the authors arrive at unfounded conclusions. We shall, here, take the opportunity to address these issues and clarify what can be legitimately concluded based on data presented in Kampf et al. First, we must take issue with the Kamp et al. interpretation of “manufacturer’s recommended” application volume. The authors reference a conference abstract by Edmonds et al. , which showed that two ABHR based on 70% Ethanol both met the in-vivo efficacy requirements of the FDA when tested at a 1.1-ml application volume, but omitted reference to a recent publication by Macinga et al.  that explained the context for the 1.1-ml application volumes. The relevant fact is, the 1.1-ml application volumes were selected for testing by Macinga et al. because these were the measured outputs of the manufacturer’s touch-free dispensers. The stated purpose of the Macinga et al. study was to evaluate the efficacy of the ABHRs at the dispensed volume. Importantly, then, this study provided real-world relevance, as it demonstrated the efficacy of products as used by workers in a healthcare environment. Moreover, it is, to-date, the only study published that provides efficacy data for products tested at dispensed volumes. It may be argued plausibly that the volume delivered from a manufacturer’s dispenser represents their recommended application volume. However, Kampf et al. fail to mention that the measured output of the US touch-free dispenser for the 85% w/w ethanol gel evaluated is 1.0 ml. Instead, they arbitrarily choose to evaluate this product at a volume of 2 ml; that is, an amount equivalent to two actuations of the touch-free dispenser. It is well known that ABHR efficacy is influenced by application volume, so it is not at all surprising that the 85% w/w ethanol product achieved higher log10 reductions than did those tested at 1.1-ml volumes. This data comparison is quite misleading, because it involves an unstated assumption that end users would disinfect hands with two actuations of the 85% ethanol gel dispenser, but only one from the 70% ethanol dispensers. Of note, in a recent study of approximately 27 million dispenser actuations, Hines et al. report that end users actuated a single dose from wall-mounted dispensers 90% of the time, despite an output of only 0.75 ml . Furthermore, we are not aware of any obvious verbiage on the dispensers for the 85% w/w ethanol gel in hospitals that instruct the end user to use two doses from the dispenser. Finally, as demonstrated in Edmonds et al.,  the 85% w/w ethanol gel at a 2-ml volume failed to meet the FDA’s efficacy criteria when tested specifically according to the methodology required by the FDA. In fact, those results suggested that the user would need to actuate the dispenser at least three times to get an amount of 85% w/w gel sufficient to meeting those criteria. We will discuss later in more detail this discrepancy between the Edmonds et al. data and the data presented in Kampf et al. Secondly, we contrast the methods used by Kampf et al. with the testing methodology required by the FDA. Specifically, Kampf et al. failed to evaluate the test products according to the required FDA method . Products were, instead, evaluated using the methodologies of ASTM E1174-06  and ASTM E2755-10 . Although each of these methods is more recent in origin than the method published in the FDA regulations, neither has been accepted for providing pivotal data to the FDA, and therefore, data from neither can be used as the basis for claims that products “meet” or “do not meet” FDA efficacy requirements. Furthermore, ASTM E-1174-06 was replaced by a revised version in 2013  and is no longer an active standard. As Kampf et al. acknowledge (in subscript to a table), “methodological differences may explain conflicting results.” It is an absolute truism that the observed efficacy of a product will vary when different test methods are used. This is precisely the reason why it is inappropriate for Kamp et al. to make statements regarding whether products meet FDA efficacy requirements when the relevant FDA method was not used. Of note, in a response to a letter to the editor submitted by a coauthor on the Kampf et al. paper,  we provided clarification of the FDA’s test method requirements, which, in this instance, they have chosen to ignore . We have no issue with Kampf using ASTM E2755-10 to test ABHR, as we have previously published showing the advantages of that method versus E1174 . However, efficacy criteria relevant to testing executed according to E2755-10 have yet to be established, so relative performance of products should only be compared using statistics. Kampf et al. do not perform any statistical analyses, so no meaningful conclusions can be drawn regarding differences in product efficacies. Thirdly, we would like to comment on the conclusions made by Kampf et al. in the context of the actual FDA efficacy requirements. Despite using methods currently not accepted by the FDA, Kampf et al. make conclusions regarding products “meeting FDA efficacy requirements” or not. The FDA requires that products be evaluated not just after a single use, but also after ten consecutive uses, to ensure they remain efficacious under high-frequency, repeated use typical of the healthcare environment. Edmonds et al. used the FDA-specified method to compare the 85% ethanol gel to the 70% ethanol gel and foam at equal application volumes (2 ml) . In that study, the 85% ethanol gel failed to meet the FDA efficacy requirement following Application 10, and efficacy was statistically inferior to that of both the 70% ethanol gel and the 70% ethanol foam. Is it possible that Kampf et al. excluded Application 10 data from their paper due to poor performance of the 85% ethanol product relative to FDA efficacy requirements and that of the 70% ethanol gel and/or foam at Application 10? Finally, we must point out that the method used by Kampf et al. to evaluate hand coverage is technically flawed. The authors state that each product was supplemented with 1.96% fluorescent dye, and hands were evaluated under UV light to determine degree to which hands were covered. However, this procedure will inherently result in a greater fluorescent signal when a greater volume of product is applied. Furthermore, the authors provide no evidence to show that reduced fluorescent signal on a particular area of the hand correlates with inadequate ABHR coverage. In fact, Macinga et al. demonstrated that a 1.1-ml application volume was sufficient to keeping hands wet for more than 20 seconds, which is within the range of 20s-30s specified in the WHO guidelines [3, 13]. Furthermore, and more importantly, the 1.1-ml application volume was sufficient to covering the hands such that the FDA efficacy requirements were met following both the 1st and the 10th application. In summary, there are multiple issues of concern in the study conducted by Kampf et al, and the conclusions are unjustified and misleading to the reader. It is disingenuous to designate the dispensed volume of two test products the “recommended dose,” but portray a volume equivalent to twice the dispensed volume to be the “recommended dose” for the third, particularly when dispensers of the latter provide no guidance for the end user that two doses should be used. To compare efficacy of products fairly, they should be tested in a single study either at equivalent volumes (as was done in Edmonds et al.), or at the volumes actually dispensed from the manufacturer’s dispensers. And, it is entirely inappropriate to claim that product efficacy does or does not satisfy FDA efficacy criteria when the appropriate test method was not used, and products were not tested for the required number of applications. At best, all that can be concluded from Kampf et al is that a 2-ml volume of one ABHR produced greater log10 reductions than did 1.1-ml volumes of two other ABHR, statistical significance unknown. As other researchers have previously demonstrated that applied volume is a critical determinant of antimicrobial efficacy, this is certainly no novel finding. Unfortunately, Kampf et al. have attempted to clarify recommendations for effective ABHR application volumes, but they have only contributed confusion. Sincerely, David R. Macinga, PhD, Research Fellow, Microbiology and Clinical Science and Sarah L. Edmonds, MS, Senior Scientist, Clinical Science References Kampf G, Ruselack S, Eggerstedt S, Nowak N, Bashir M: Less and less-influence of volume on hand coverage and bactericidal efficacy in hand disinfection. BMC Infect Dis 2013, 13:472. Edmonds S, Macinga DR, Paulson D: The influence of ABHR product format on in vivo efficacy: a meta-analysis. Am J Infect Contr 2012, 40:e43. Macinga, D., Edmonds, S., Campbell, E., Shumaker, D., Arbogast, J.: Efficacy of Novel Alcohol-Based Hand Rubs at Typical “In Use” Volumes. Infect Contr Hosp Epidemiol, 2013 34:299-301. Hines J, Alper P, Voss A, McGeer A: P101: Product dose considerations for real-world hand sanitiser efficacy. Antimicrob Res Infect Contr 2013, 2(Suppl 1):P101. Edmonds SL, Macinga DR, Mays-Suko P, Duley C, Rutter J, Jarvis WR, Arbogast JW: Comparative Efficacy of Commercially Available Alcohol-Based Hand Rubs and WHO-Recommended Hand Rubs: Formulation Matters. Am J Infect Control 2012, 40:521-525. Food and Drug Administration. Tentative final monograph for healthcare antiseptic drug products; proposed rule. Federal Register 1994 59:31441-31450. ASTM International. E-1174-06: Standard Test Method for Evaluation of the Effectiveness of Health Care Personnel or Consumer Handwash Formulations. West Conshohocken, PA: ASTM International, 2006 ASTM International. E-2275-10: Standard Test Method for Determining the Bacteria-Eliminating Effectiveness of Hand Sanitizer Formulations Using Hands of Adults. West Conshohocken, PA: ASTM International, 2010. ASTM International. E-1174-13: Standard Test Method for Evaluation of the Effectiveness of Health Care Personnel or Consumer Handwash Formulations. West Conshohocken, PA: ASTM International, 2013. 10. Eggerstedt S: Letter to the editor on "Comparative efficacy of commercially available alcohol-based hand rubs and World Health Organization-recommended hand rubs". 2013. Am J Infect Contr 41:472-4. 11. Edmonds SL, Macinga DR, Jarvis WR: Reply to letter to the editor on "Comparative efficacy of commercially available alcohol-based hand rubs and World Health Organization-recommended hand rubs". 2013. Am J Infect Contr 41:474-5. 12. Macinga DR, Beausoleil CM, Campbell E, Mulberry G, Brady A, Edmonds SL, Arbogast JW: Quest for a realistic in vivo test method for antimicrobial hand-rub agents: introduction of a low-volume hand contamination procedure. Appl Environ Microbiol 2011, 77:8588-94. 13. World Health Organization. Who guidelines on hand hygiene in health care. Geneva, Switzerland: World Health Organization, 2009. Competing interests We are both employees of GOJO Industries, a manufaturer of skin care products including alcohol based hand rubs. We both work in Research and Development.