Table 4 Outcome Measures using RE-AIM

Reach

• How many patients can be reached by the interventions?

• Number of AOM episodes

• Electronic health record (EHR) data (baseline, quarterly during intervention, post-intervention)

Effectiveness

• Do the interventions increase parent satisfaction?

• Do the interventions reduce antibiotic use for AOM?

• % very or extremely satisfied • % children take antibiotic (yes/no)

• Parent surveys (0, 10 days after diagnosis, throughout intervention)

• Parent focus groups (pre- and post-intervention)

Adoption

• How many clinics will adopt the interventions?

• Do the interventions require adaptations to meet local needs?

• Number/proportion of eligible clinics that agree to participate

• Characteristics of clinics that participate vs. clinics that opt not to participate

• Number/proportion of enrolled clinics that implement all program components

• AC, DHHA, and IH health system data (pre-intervention, post-intervention)

• Clinician and administrator surveys (post-intervention)

• Key informant qualitative interviews (pre intervention)

Implementation

• How many clinicians participate in intervention components?

• Are the interventions acceptable to clinicians?

• How effectively is SDM^a used?

• How many children are managed with watchful waiting v immediate prescription?

• How many parents fill antibiotic prescriptions (even if not taken by child)?

• Implementation of intervention components and time to implementation

• Use of program components (fidelity)

- Education sessions

- Feedback reports

- EHR tools

- Clinician champion

- Meet CME program requirements

• Clinician perceptions of intervention component utility, acceptability, and appropriateness

• Parent perception of SDM and appropriateness

• % managed by immediate antibiotic

• % parents fill antibiotic prescription

• Stages of Implementation Completeness [47] (SIC) tracking (quarterly through the study)

• Attendance at education sessions, meeting CME Requirements, proportion of AOM episodes where EHR tools were used, proportion of feedback reports read by clinicians- tracked using read receipts [48] (quarterly during intervention and post-intervention)

• Clinician and administrator surveys using the acceptability of intervention measure (AIM) and intervention appropriate measure (IAM) (post intervention [34]

• Focus groups of parents (pre- and post-intervention)

Maintenance

• What is the likelihood of sustainability of the interventions?

• Do clinicians and administrators think it would be feasible to implement the interventions in other settings?

• Could the interventions be implemented in other settings?

• Clinical Sustainability Assessment Tool (CSAT) Sustainability Score [49]

• Feasibility of intervention (FIM) measure [50]

• Clinician and administrator surveys using CSAT and feasibility of intervention measure (FIM) [50] (post-intervention)