Using the 4 pillars™ practice transformation program to increase adult influenza vaccination and reduce missed opportunities in a randomized cluster trial

Background An evidence-based, step-by-step guide, the 4 Pillars™ Practice Transformation Program, was the foundation of an intervention to increase adult immunizations in primary care and was tested in a randomized controlled cluster trial. The purpose of this study is to report changes in influenza immunization rates and on factors related to receipt of influenza vaccine. Methods Twenty five primary care practices were recruited in 2013, stratified by city (Houston, Pittsburgh), location (rural, urban, suburban) and type (family medicine, internal medicine), and randomized to the intervention (n = 13) or control (n = 12) in Year 1 (2013-14). A follow-up intervention occurred in Year 2 (2014-15). Demographic and vaccination data were derived from de-identified electronic medical record extractions. Results A cohort of 70,549 adults seen in their respective practices (n = 24 with 1 drop out) at least once each year was followed. Baseline mean age was 55.1 years, 35 % were men, 21 % were non-white and 35 % were Hispanic. After one year, both intervention and control arms significantly (P < 0.001) increased influenza vaccination, with average increases of 2.7 to 6.5 percentage points. In regression analyses, likelihood of influenza vaccination was significantly higher in sites with lower percentages of patients with missed opportunities (P < 0.001) and, after adjusting for missed opportunities, the intervention further improved vaccination rates in Houston (lower baseline rates) but not Pittsburgh (higher baseline rates). In the follow-up intervention, the likelihood of vaccination increased for both intervention sites and those that reduced missed opportunities (P < 0.005). Conclusions Reducing missed opportunities across the practice increases likelihood of influenza vaccination of adults. The 4 Pillars™ Practice Transformation Program provides strategies for reducing missed opportunities to vaccinate adults. Trial registration This study was registered as a clinical trial on 03/20/2013 at ClinicalTrials.gov, Clinical Trial Registry Number: NCT01868334, with a date of enrollment of the first participant to the trial of April 1, 2013.


Background
Adult influenza vaccination rates in the United States (U.S.) continue to languish at approximately 44 % as of 2014-2015 reported estimates [1], with annual increases hovering at less than 2 percentage points [2,3]. The reasons for the disparity between reported rates in the community, the desired rate of 70 % [4] set forth in the Healthy People 2020, and meaningful annual increases are legion. They include logistical issues at the practice level such as storage, cost [5], return policies for unused vaccine, and choice of vaccines; strongly held personal beliefs about influenza vaccine at the patient level such as belief that the vaccine is not necessary, effective or safe [6]; and broader issues such as, changes in year-toyear effectiveness, timing of vaccine distribution [7] and vaccine administration reimbursement.
Several system-level efforts have been undertaken or implemented that have attenuated some of the barriers to vaccination. The Affordable Care Act requires that certain preventive services including immunizations be covered as part of basic care [8]. This mandate should effectively eliminate patients' financial barriers to receiving influenza vaccine. The Centers for Disease Control and Prevention (CDC) encourages vaccine distribution policies that are designed to reduce regional and local shortages of vaccine, i.e., partial orders are shipped nationwide and orders are completed later as more stock becomes available [7]. An increasing number of hospitals and health systems now require their employees to receive influenza vaccine [9,10].
To have any hope of attaining the Healthy People 2020 goal, a business-as-usual approach to increasing influenza vaccine uptake is no longer acceptable. A single strategy is unlikely to be successful. The Task Force on Community Preventive Services has recommended multi-strategy, evidence-based interventions [11] as an effective means of increasing immunization rates. These interventions should enhance access to vaccination services, increase community demand for vaccines, and improve provider-or system-based interventions.
The 4 Pillars™ Practice Transformation Program, also known as the 4 Pillars™ Immunization Toolkit, (4pillarstoolkit.pitt.edu) is a compilation of evidence-based best practices for increasing immunizations in primary care settings. It is built on decades of research by the investigators into the barriers to and facilitators of adult immunizations from the provider and patient perspectives, and trials of successful strategies. The 4 Pillars™ Program was the foundation of an intervention implemented in a randomized controlled cluster trial (RCCT), to increase adult immunization rates and reduce missed opportunities to vaccinate among patients of primary care practices in Pittsburgh and Houston [12]. The purpose of this study is to report on changes in adult influenza immunization rates and on factors related to the likelihood of receipt of influenza vaccine.

Sample size and sites
Optimal Design software (University of Michigan, Version 1.77, 2006) was used to calculate sample size for a randomized trial seeking a 10-15 % absolute increase in vaccination rate with a minimum practice size of 100 patients. A sample size of 20 clusters or sites (10 Intervention and 10 Control practices) was determined to be necessary to achieve 80 % power with an alpha of 0.05. Eligible primary care family medicine (FM) and internal medicine (IM) practices from a practice-based research network (PBRN) in Pittsburgh (FM Pittnet), a clinical network in Southwestern Pennsylvania (UPMC Community Medicine, Inc.) and a PBRN in Houston (SPUR-Net) were solicited for participation. All Houston sites were publicly funded, safety net practices, caring for a disadvantaged population; whereas, Pittsburgh practices were smaller private practices or residency sites with patients from across the socioeconomic spectrum. When 25 sites agreed to participate, solicitation ceased. All sites used a common electronic medical record (EMR), EpicCare within their respective health systems.

Cluster randomization
Cluster randomization allocates clinical practices rather than individuals to the intervention arms [13]; thus, each site or office (some practices had more than one) was considered as a cluster. Eligibility requirements included having a significant adult practice, preliminary baseline vaccination rates for at least one adult vaccine <50 % and a willingness to make office changes to increase influenza, pneumococcal and Tdap vaccination rates. Participating practices were stratified first by metropolitan area (Pittsburgh or Houston), then in Pittsburgh only, by location (urban, suburban or rural) and by discipline (IM or FM). The practices were then randomized into the intervention or control arms within strata (Fig. 1).
Year 2 control practices were informed that their intervention would take place the following season and were not contacted again until the next year.

Pillars™ program
The 4 Pillars™ Practice Transformation Program is founded on four evidence-based [11,14,15] key domains: Pillar 1 -Convenient vaccination services; Pillar 2 -Communication with patients about the importance of immunization and the availability of vaccines; Pillar 3 -Enhanced office systems to facilitate immunization; and Pillar 4 -Motivation through an office immunization champion (Champion). Table 1 describes the primary strategies contained in the 4 Pillars™ Program. The 4 Pil-lars™ Program includes background on the importance of protecting patients against vaccine-preventable diseases, barriers to vaccination from both provider and patient perspectives and strategies to eliminate those barriers. Practices were expected to implement strategies from each of the 4 pillars.
The 4 Pillars™ Program was provided as a printed and bound document, supplemented by a web-based practice transformation dashboard. The dashboard was developed from the work of Fixsen et al. [16], who established an empirically-based implementation framework that includes systematic uptake, establishment, and maintenance of research findings into routine practice. The core components include: staff selection and training on the specific evidence-based practices, expert consultation and coaching of staff and administration, program evaluation to assess and provide feedback, facilitative administrative supports to ensure data are used to focus and inform decision making, and systems interventions. Once the practice was registered, any staff member could log into the dashboard. The Champion was responsible for registering the practice and its staff members, and identifying strategies that the practice would implement. The 4 Pillars™ Program provided step-bystep guidance for implementing the strategies, and the dashboard showed the practices' progress through the change process. Practices could monitor their progress on graphs that reported biweekly numbers of vaccines given.

Interventions
The intervention was designed using Diffusion of Innovations theory [17], and included the 4 Pillars™ Practice Transformation Program, provider education, and oneon-one coaching of the immunization champion for each practice. Two of the investigators (AEB, MPN) visited the intervention sites each year to introduce the study and the 4 Pillars™ Program and to work with staff to develop practice-specific ideas for implementing strategies.
Each practice was asked to identify a Champion who would be responsible for updating the practice transformation dashboard as intervention strategies were employed to guide strategy implementation. Other tasks for the Champion included participating in the biweekly telephone call with a research liaison for coaching, ensuring that chosen strategies were being implemented, and working to maintain motivation of the staff. Each practice was given a graph showing biweekly progress towards their goal based on an overall 20 % increase over the previous year's total adult influenza vaccines given. These graphs were to be used by the Champion to encourage the staff to maintain motivation or stimulate additional changes to increase vaccination rates. The research liaison discussed these graphs with the Champions during their calls or visits.

Data collection
De-identified demographic, office visit and vaccination data were derived from EMR data extractions performed by the UPMC Center for Assistance in Research using the eRecord and from a similar EMR extraction by staff of the SPUR-NET for the Houston sites. A longitudinal data base was created with only those patients who had a visit each year during all three years, creating a cohort of individuals who would have been patients of the practice during the entire study period.

Statistical analyses
Descriptive analyses were performed for patient demographic characteristics (age, sex, race, ethnicity, and health insurance). Because of significant differences in patient populations, size and structure of the practices in Houston and Pittsburgh, the respective sites were grouped separately for analysis. Age was used as a continuous variable. Race and ethnicity were recorded differently in each city. In Pittsburgh sites, with few Hispanic patients, ethnicity was rarely recorded; hence patients were grouped by race into white and non-white with blacks and Hispanics assigned to the non-white group and only race data are presented and used in analysis. In Houston sites, with few non-Hispanic patients, race was rarely recorded; hence only ethnicity (Hispanic and non-Hispanic) is presented and used in analysis. Proportions were reported for categorical variables and means and standard deviations were reported for continuous variables.
The primary outcome measure, influenza vaccination rate was reported at the end of baseline  • Offer open access/walk-in vaccination during office hours.
• Hold express vaccination clinics outside normal office hours where only influenza vaccine is offered and systems for check-in, screening, and record keeping are streamlined.
• Create a dedicated vaccination station.
• Extend the influenza vaccination season by vaccinating as soon as supplies arrive and continuing to vaccinate as long as flu is circulating in the community.

Pillar 2 Communication with patients about the importance of vaccination and the availability of vaccines
• Train staff to discuss influenza vaccine during routine processes such as vital signs • Discuss the serious nature of influenza • Promote vaccination of staff to set a good example • Record telephone on-hold messages that advertise vaccine availability or promote vaccination.
• Use posters/fliers/electronic message board/website postings/social media promoting vaccination

Pillar 4 Motivation through an office immunization champion
• Create a chart to track progress. Set an improvement goal and regularly track progress (e.g., daily or weekly). Post the graph of your progress in a prominent location and update it regularly.
• Provide ongoing feedback to staff on vaccination progress using email, posted notices, making announcements, or using a combination of these. Encourage, nudge, and cheer as needed to keep up the momentum.
• Report upon progress at staff or huddle meetings. Facilitate discussion at these meetings to identify which pillar activities are working, which are not working and why, and to identify changes that need to be made.
• Create a competitive challenge among your staff for the most vaccinations given.
• Provide rewards for successful results to create a fun-spirited environment that promotes vaccination across the practice. Ideas include: reward for highest vaccinator, team competitions, vaccination goal poster contest, etc.
six practices that did not actively participate in Year 2 were referred to as the maintenance group. In Houston, the Year 1 intervention sites were referred to as the maintenance group and the Year 1 control sites that received the intervention in Year 2 were referred to as the active intervention group. The August through January dates were used because the new seasonal influenza vaccines typically begin to arrive at practices in August and the majority of adult influenza vaccines are given by the end of January. Missed opportunities for all patients who were seen during the influenza vaccination seasons (8/1/2013-1/31/ 2014 and 8/1/2014-1/31/2015) were determined to be all visits in which an influenza vaccine was not received until vaccination or the end of the influenza season (1/ 31). The visit in which an influenza vaccine was given and post vaccination visits were not counted. Patients who were not seen at all during the influenza vaccination  and active intervention and maintenance for the Year 2 pre-post study for influenza vaccination rates and proportion of the patients with ≥1 missed opportunity [18]. Chi-square tests were used to compare the PP changes from baseline to follow up between groups.
To determine which factors were related to influenza vaccination rates, while accounting for the clustered nature of the data, generalized estimating equation (GEE) models were used to examine the likelihood of influenza vaccination after the intervention period, taking account of heterogeneity in demographic characteristics (including age, sex, and race/ethnicity and health insurance status) as well as the site -level variables intervention arm and proportion of patients with missed opportunities. An unstructured correlation matrix was used to accommodate the within-patient variation due to repeated annual measurement of influenza vaccination. Statistical significance of two-sided tests was set at type I error (alpha) equal to 0.05. All analytical procedures were performed using SAS® 9.3.

Results
Twenty-four sites completed the Year 1 intervention (one site dropped out); their demographic and other characteristics are shown in Table 2. Houston sites were part of a publicly funded safety net system which were established to care for uninsured/underinsured patients; whereas, Pittsburgh sites were both training/residency sites and private practices. Houston sites were larger with higher proportions of Hispanic patients, female patients, and non-commercially insured patients than Pittsburgh sites. For these reasons, the results are presented separately for each city. Table 3 shows the influenza vaccination rates for each site and each intervention group for the baseline and intervention periods. Vaccination rates ranged from a low of 23.6 % to a high of 62 % at baseline across all sites. During the intervention, Pittsburgh intervention sites significantly increased influenza vaccination rates an average of 5.0 PP (P < 0.001), while control sites significantly increased influenza vaccination an average of 6.5 PP (P < 0.001). In Houston, intervention sites significantly increased influenza vaccination an average 2.7 PP, while control sites increased influenza vaccination an average of 5.2 PP (P < 0.001). Influenza vaccination increased more in both the control groups than intervention groups (P < 0.001). At the same time, the percent of patients with at least one missed opportunity to vaccinate decreased in the Pittsburgh intervention and control groups and the Houston control group (P < 0.001).

Year 1 -RCCT
In GEE regressions (Table 4), in the Pittsburgh sites, which had higher baseline rates, likelihood of influenza vaccination was significantly higher for females, older patients, white patients and those with commercial insurance or Medicare. While those in the intervention group were not more likely to receive influenza vaccine, those patients in practices having fewer patients with at least one missed opportunity were more likely to receive the vaccine (P < 0.05). In the Houston sites, the likelihood of influenza vaccination was significantly higher among those who were older, of Hispanic ethnicity, in the intervention group and in practices with fewer missed opportunities (P < 0.05).
Year 2pre-post study During the Year 2 pre-post study, vaccination rates (Table 5) among the active intervention groups in the pre-intervention period were 54.5 % for Pittsburgh and 44.7 % for Houston; whereas vaccination rates for the maintenance groups were 55.2 % for Pittsburgh and 40.2 % for Houston, at the beginning of their maintenance period. At the end of the Year 2 intervention period, both the active intervention and maintenance groups significantly increased influenza rates and decreased proportion of patients with missed opportunities in both cities. In bivariate analyses, the change in influenza vaccination did not differ between groups, in Pittsburgh, but was significantly higher in the active intervention group than the maintenance group in Houston (P < 0.001) In GEE regression analyses (Table 6) in Pittsburgh, older age, being female, as well as being in the intervention group and being a patient at a site with fewer missed opportunities were related to increased likelihood of influenza vaccination (P < 0.05). Similarly in Houston, . A decrease in missed opportunities is the desired outcome * P value is a two-sided probability from Cochran-Armitage trend test for difference between pre and post intervention (P < 0.001) **P value is a two-sided probability from Cochran-Armitage trend test for difference between pre and post intervention (P < 0.05) † P value is a two-sided probability from chi-square test of PP differences between intervention arms older age, being female, Hispanic, in the intervention group and in a site with fewer missed opportunities were all related to increased likelihood of influenza vaccination (P < 0.01).

Discussion
Secular trends in U.S. adult influenza vaccination rates indicate a slow increase in vaccination of approximately 2 percentage points per year. At this rate, it would take nearly 15 years to achieve the national goal of 70 % [4], given the distance between that goal and the current national rate of 42 % [19]. In our randomized controlled cluster trial, all groups increased vaccine uptake by 2.7-6.5 %, suggesting that the intervention using the 4 Pil-lars™ Practice Transformation Program was not more effective than secular trends. Variation in the level of improvement among practices was noted, with some sites making little to no improvement. Research has shown that impediments to successful quality improvement projects include not having allocated time to devote to the project, lack of leadership support, not having a performance assessment and not having a project champion [20]. In this study, some Champions did not hold positions of influence in the practice and/or were not allowed work time to use the online 4 Pillars™ Program and dashboard to take advantage of the resources and guidance provided therein. Other issues that may have hindered practice change were 1) the lead physician in one small practice was nearing retirement, hence was not engaged in the effort; b) one site was located in a rural Amish community with high vaccine refusal rates; 3) late delivery of influenza vaccine in Year 1 in the Pittsburgh sites; and 4) feedback to the sites on immunization progress was delayed. When the control groups were offered the intervention and the intervention groups were in maintenance, small additional gains in influenza vaccination rates were realized in the active intervention groups, with no loss among maintenance groups, suggesting that behavior changes to improve vaccination were persisting.
Research has shown that missed opportunities to vaccinate are frequently associated with low vaccination rates [21][22][23][24]. In one study, among unvaccinated high risk adults, 90 % reported at least one visit in which influenza vaccine could have been administered [25]. The regression analyses in this study supported those findings and indicated that reducing missed opportunities is a critical element in increasing influenza vaccination. The intervention increased the likelihood of influenza vaccination when missed opportunities decreased in the practices. The 4 Pillars™ Program recommends standing order protocols for clinical staff to offer vaccines, reviewing vaccination status at every visit and offering express vaccine services such as influenza vaccine-only clinics, and walk-in vaccinations during influenza vaccination season. Consistent use of these strategies should reduce the number of missed opportunities and in turn increase vaccination rates [11,26]. We believe that routine assessment of vaccination status that triggers standing orders is a powerful combination (Pillar 3).

Strengths and limitations
The strengths of this study are its randomized design, the large number and diversity of patients and practice settings including safety net clinics, and two intervention years of vaccination reporting. These factors enhance its generalizability. The study's limitations include late delivery of influenza vaccine in Year 1 in the Pittsburgh sites that may have diminished the intervention effect. During the first intervention year, delivery of the EMR data was delayed, preventing the research team from providing feedback about their progress to the sites in both cities early in the intervention. Increases in vaccinations in the control arm when those sites were not in an intervention group may be due to a Hawthorne effect or transference from the intervention and have been reported in other studies [27]; secular trends are smaller than the changes noted in this study.

Conclusions
As the national adult influenza vaccination rate continues to creep towards national goals, strategies are needed to jumpstart efforts to increase the rate of improvement. An intervention that includes the 4 Pillars™ Practice Transformation Program can assist primary care practices with reducing missed opportunities to vaccinate thus increasing adult influenza vaccination rates.