Population effectiveness of the pentavalent and monovalent rotavirus vaccines: a systematic review and meta-analysis of observational studies

Abstract

Background

Rotavirus was the leading cause of acute gastroenteritis (AGE) in infants and young children prior to the introduction of routine vaccination. Since 2006 there have been two licensed vaccines available; with successful clinical trials leading the World Health Organization to recommend rotavirus vaccination for all children worldwide. In order to inform immunisation policy we have conducted a systematic review and meta-analysis of observation studies to assess population effectiveness against acute gastroenteritis.

Methods

We systematically searched PubMed, Medline, Web of Science, Cinhal and Academic Search Premier and grey literature sources for studies published between January 2006 and April 2014. Studies were eligible for inclusion if they were observational measuring population effectiveness of rotavirus vaccination against health care attendances for rotavirus gastroenteritis or AGE. To evaluate study quality we use used the Newcastle-Ottawa Scale for non-randomised studies, categorising studies by risk of bias. Publication bias was assessed using funnel plots. If two or more studies reported a measure of vaccine effectiveness (VE), we conducted a random effects meta-analysis. We stratified analyses by World Bank country income level and used study quality in sensitivity analyses.

Results

We identified 30 studies, 19 were from high-income countries and 11 from middle-income countries. Vaccine effectiveness against hospitalization for laboratory confirmed rotavirus gastroenteritis was highest in high-income countries (89% VE; 95% CI 84-92%) compared to middle-income countries (74% VE; 95% CI 67-80%). Vaccine effectiveness was higher for those receiving the complete vaccine schedule (81% VE; 95% CI 75-86%) compared to partial schedule (62% VE; 95% CI 55-69%). Two studies from high-income countries measured VE against community consultations for AGE with a pooled estimate of 40% (95% CI 13-58%; 2 studies).

Conclusions

We found strong evidence to further support the continued use of rotavirus vaccines. Vaccine effectiveness was similar to that reported in clinical trials for both high and middle-income countries. There is limited data from Low income settings at present. There was lower effectiveness against milder disease. Further studies, should continue to report effectiveness against AGE and less-severe rotavirus disease because as evidenced by pre-vaccine introduction studies this is likely to contribute the greatest burden on healthcare resources, particularly in high-income countries.

Background

Prior to the introduction of rotavirus vaccine into childhood immunisation schedules, rotavirus was the most common cause of severe gastroenteritis in infants and young children. Thus, rotavirus gastroenteritis (RVGE) was estimated to be responsible for 453,000 deaths worldwide in children under 5 years of age in 2008, with over 90% of deaths occurring in low-income countries [1]. The global morbidity from rotavirus infection was also substantial with pre-vaccine introduction studies indicating that approximately 40% of diarrhoeal hospitalisations in children were caused by rotavirus [2]. In middle and high income countries without vaccination the burden of RVGE remains substantial in infants and young children with high rates of disease and RV the major contributor to diarrhoea hospitalisation. In the UK prior to vaccine introduction RVGE was estimated to be responsible for 45% of acute gastroenteritis hospital admissions, 80,000 primary care consultations and 750,000 annual diarrhoeal episodes in children under 5 years of age [3, 4]. In middle income countries such as, Mexico and Peru, prior to vaccine introduction the average incidence of RVGE was 0.3 episodes per child per year in children <2 years, resulting in significant healthcare use and mortality [5]. Although the large majority of severe RVGE occurs among young children, older children and adults can be affected, however rotavirus infection often causes milder symptoms or is asymptomatic in these ages, meaning the true burden and rate of disease incidence is poorly understood.

Since improvements in sanitation and hygiene are not expected to reduce the incidence of rotavirus infection, and treatment of RVGE is limited to rehydration therapy, immunisation of infants is considered the best option for control of the global burden of rotavirus disease. Since 2006 there have been two live-attenuated oral rotavirus vaccines that are licensed for use globally. A two dose monovalent vaccine (Rotarix®, GlaxoSmithKline Biologicals, Belgium), with the first dose typically administered at between 6-8 weeks and a second dose at least 4 weeks later and a three dose pentavalent vaccine (RotaTeq®, Merck), administered at 6-12 weeks of age with subsequent doses at 4-10 week intervals. Randomised controlled trials (RCTs) demonstrated both vaccines to be efficacious against severe RVGE; vaccine efficacy of over 80% has been shown in middle and high-income countries, whilst trials in low-income settings have reported efficacy against severe RVGE of 40-60% [6]. These trials led to a World Health Organisation recommendation for universal vaccination of all children [1, 7]. More than 90 countries have since adopted rotavirus vaccination and the global mortality from RVGE estimated to have fallen to 215 000 in 2013 with almost 50% of deaths occurring in four lower-income countries [8, 9]. Currently within the European Union only nine countries include rotavirus vaccination in their childhood immunisation programme [10, 11].

It is now a decade since the licensing and first introduction of rotavirus vaccination into childhood vaccination schedule. In order to inform immunisation policy, we have conducted a systematic review and meta-analysis of the literature on observational studies in order to assess the population effectiveness of the Rotarix® and RotaTeq® against rotavirus gastroenteritis. Effectiveness was examined by severity and by region.

Methods

Inclusion/exclusion criteria

We included prospective or retrospective observational studies (cohort and case-control studies) reporting the population effectiveness of the monovalent Rotarix® (RV1) or pentavalent RotaTeq® (RV5) against healthcare attendance for RVGE or other acute gastroenteritis (AGE), in countries where the vaccines are included in the national immunisation programme or privately offered through medical insurance. Studies published between January 2006 and 28^th April 2014 were eligible for inclusion. Review articles, editorials and conference abstracts were included in citation checking but excluded from final analysis. Randomised controlled trials were also excluded.

Search strategy

We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. We systematically searched PubMed, Medline, Web of Science, Cinhal and Academic Search Premier, OpenGrey and the Cochrane Library databases using a well-defined search strategy following a protocol registered on the University of York database for Prospectively Registered Systematic Reviews (PROSPERO: 2014:CRD42014012974). A number of relevant organisations websites were also systematically searched, and included the World Health Organisation, Public Health England, and Centers for Disease Control and Prevention. We systematically searched the literature by, pairing the terms [vacc*] and [rotavirus] with the following key words: [immuni*], [effect*], [ evaluation], and [efficacy].

Authors (DH and CM) replicated the search strategy and independently screened titles and abstracts to identify full studies that were eligible for full publication review. Subsequently these two authors independently assessed the full text publications and their final inclusion was based on a consensus between the reviewers (DH, CM, KS, AT).

Data extraction

Data extraction was autonomously carried out by three authors (DH, AT, KS) and a collaborator (MSC) using a pre-designed internally piloted extraction tool. For each study the following information was extracted: Author, Year of publication, country and region of study, funding source, study period, country vaccine coverage, study type, sample size, age of subjects, type of vaccine (RV1 and / or RV5) in case and controls groups, case definition, control definition, number of vaccine doses, relative risks / risk ratios (RR) or odds ratio (OR) or vaccine effectiveness (VE) and 95% confidence intervals (95% CI) and, if applicable, a measure of intussusception.

Grading of selected studies

The Newcastle-Ottawa Scale (NOS) for assessing the quality of non-randomised studies was identified as an appropriate tool to assess study quality [12]. Case-control and cohort-studies were assessed using the tool by the same three researchers that carried out data extraction. To quality assess case-control studies the scale used: 1) adequate definition of a case and the representativeness of cases; 2) controls selection and case definition; 3) matching of controls or adjustment for confounders in analysis; 4) ascertainment of method of cases and controls in terms of exposure (rotavirus vaccination) and non-response rate. To quality assess cohort-studies the factors assessed were: 1) representativeness of the vaccinated cohort and selection of unvaccinated cohort in relation to the vaccinated; 2) ascertainment of vaccination record, confirmation of rotavirus negative at start of study; 3) matching of exposed and non-exposed in design or adjustment for confounders in analysis; 4) ascertainment of outcome (rotavirus infection); 5) follow-up duration in relation to outcome appearance (e.g. 1 year from vaccination date) and was follow-up adequate (we defined adequate as ascertainment of outcomes for >80% of participants). The scale is categorised into three groups, selection, comparability and outcome/exposure; a maximum of nine stars can be awarded to each study. Studies scoring 0 in any of the categories were classified as having a high risk of bias, studies scoring 1 in any categories (moderate risk of bias) and 2 or above in all categories (low risk of bias).

Statistical analysis

We used Stata, version 14, statistical software (Stata Corp., College Station, TX, USA) to perform all statistical calculations for this meta-analysis. Meta-analyses were conducted separately for cohort and case-control studies. We used the study published RR for cohort studies and OR for case-control studies and calculated standard errors (SE) using study reported confidence intervals in the formula:

$$ SE=\frac{\mathit{\ln}\left( Upper confidence interval\right)-\mathit{\ln}\left( Lower Confidence interval\right)}{3.92} $$

Where studies did not report OR or RR, authors calculated crude OR or RR and SE using reported numbers of cases and controls. When a study reported both unadjusted and adjusted RRs/ORs, adjusted RRs/ORs were included in meta-analysis and unadjusted estimates excluded. Vaccine effectiveness was defined as 100 × (1 − RR) or 100 × (1 − OR). A random effects model was used to provide pooled estimates of VE. Because of differences in reported vaccine efficacy a decision was taken during data extraction, for analyses to be stratified by country income category, as defined by the World Bank and measured using gross national income per capita [6, 13]. Where a study had reported VE for multiple years the estimate for mid or most recent year (if only two years) were included in the meta-analysis. Heterogeneity was measured using chi-squared (χ2) heterogeneity p-values and I ² statistics. A p-value<0.1 was considered to identify statistically significant heterogeneity rather than 0.05 due to the small number of studies included. The percentage of variance across studies due to heterogeneity rather than chance was categorised as low, moderate and high using I ² values of 25-49%, 50-74% and >=75%, respectively [14].

Sensitivity analysis was conducted based on the NOS score, excluding studies with a high or moderate risk of bias and assessing whether a study was conducted in a country with routine vaccination (part of recommended health policy) or in countries where vaccination provision is private or only available in some states. Subgroup analysis was conducted on: number of doses (1 dose, and full doses), age group, and vaccine type. Both number of doses and vaccine type were identified as important analyses post-hoc. Where studies reported had more than one type of control group the following hierarchy was used to select estimates for use in meta-analyses: 1) community/neighbourhood; 2) hospital non AGE controls; hospital RV negative AGE controls.

Publication bias was checked by funnel plot asymmetry and use of Begg’s test [15].

Results

Study characteristics

The initial search strategy identified 2,097 studies as potentially relevant; of these, 30 were eligible for inclusion in the review (Fig. 1) [16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45]. Two summary tables of study characteristics are available in Additional file 1. Seven studies were cohort studies and 23 were case-control studies (Tables 1 and 2.) Nineteen were from high-income countries and eleven were from middle income countries. Seven studies declared some funding from industry related to the rotavirus vaccines under study. Over a third of studies were conducted in the USA (n=12). The majority of studies (27/30) reported on RVGE hospitalisations and/or emergency department (ED) attendances for AGE with a positive laboratory test for rotavirus. A study by Mast et al., [16] measuring vaccine effectiveness against RVGE hospitalisation and ED attendances included only cases with severe disease defined by a Vesikari score of greater than 11 in their VE estimate [16, 46, 47]. Two studies reported RVGE ED attendances and hospitalisations combined. Five studies included community consultations for either RVGE or AGE [17,18,19,20,21], which included a range of definitions including outpatient attendance, physician consultation and telephone consultation. The study by Fontes Vieira et al. 2011 conducted in Brazil on a community cohort examined effectiveness of vaccination against laboratory confirmed RVGE but did not report an estimate of VE; a crude estimate was therefore calculated by the authors [17]. In the majority of studies, laboratory confirmation of RVGE followed hospitalisation, an ED attendance or GP consultation for gastroenteritis symptoms such as diarrhoea. Study selection identified five studies from countries (Spain and Israel) where routine childhood vaccination is not available but either the monovalent or pentavalent vaccine is available privately and / or only in some states [18, 22, 24, 31, 42].

Fig. 1

Flow chart of publications included and excluded for this review

Table 1 Cohort study vaccine effectiveness against hospitalisations, emergency department attendances and community consultations for RVGE or AGE

Table 2 Case-control study vaccine effectiveness against hospitalisations, emergency department attendances for RVGE or AGE (partial is for 1 dose of RV1 or RV5 if reported)

Controls for case-control studies were primarily hospital controls that were admitted for AGE symptoms but with a rotavirus negative laboratory test result. A few studies also used community asymptomatic controls or non-AGE hospital controls, such as children admitted with acute respiratory infection (Additional file 1: Table S2). VE was measured for a range of age groups across studies.

Quality of included observational studies

The quality of studies varied considerably (Table 3). It was difficult to ascertain for most studies whether history of disease in control subjects was considered. The majority of studies used a combination of vaccination cards and medical records to ascertain vaccination status. The majority of studies either matched controls or adjusted for age in the analysis as a minimum and those with community controls often used an indicator of residence such as GP location as a covariate. A high risk of bias was identified in two out of seven cohort studies [17, 23] and two out of 23 case-control studies had a high risk of bias [24, 25].

Table 3 Quality of observational studies included in the review of rotavirus vaccine effectiveness. Using the Newcastle-Ottawa Scale

Meta-analysis of vaccine effectiveness against hospitalisation or combined hospitalisation and emergency department attendance for laboratory confirmed RVGE

For this outcome measure cohort studies were too few to conduct a meta-analysis (Table 1). We therefore, included 21 out of the 22 case control studies that measured vaccine effectiveness against hospitalisation or hospitalisation and ED attendance for laboratory confirmed RVGE in the meta-analysis. Patel et al. 2009 was excluded because a more recent publication Patel et al., 2012 provided more recent estimates of effectiveness for the same cohort [26, 27]. If studies reported more than one age group the overall estimate for the broadest age group was included in the meta-analysis. Studies which reported on 0 doses vs full dose or 0 dose vs 1+ dose were included. Some 22 estimates from 21 studies were included, as Payne et al, 2013 had separate results for RV1 and RV5 [28]. The funnel plot shows some asymmetry, however this is not significant for Begg’s (p=0.06) tests (Fig. 2). Therefore, we included all 21 studies in the meta-analysis.

Fig. 2

Funnel plot of vaccine effectiveness against hospitalisation or hospitalisation and emergency department attendance for laboratory confirmed RVGE (only adjusted effect estimates included)

There was statistically significant heterogeneity (I ²= 60.6, p < 0.001) across all studies (Fig. 3). Therefore pooled estimates of OR (0.18, 95% CI 0.14–0.23, 21 studies [22 estimates], p<0.001) were calculated using a random effects model. The pooled VE was therefore 82% (95% CI 80–88%; 21 studies [22 estimates], p<0.001). A stratified analysis by World Bank Country Classifications calculated pooled estimated ORs. Pooled VE was lower in middle-income countries (74% VE; 95% CI 67-80%; 9 studies, p<0.001) compared with high income countries (89% VE; 95% CI 84-92%; 12 studies [13 estimates], p<0.001). There was low to moderate heterogeneity for middle (I ²= 37.4%, p=0.120) and high income countries (I ² = 40.8%, p=0.06). The study by Cotes-Cantillo et al., 2014 was the only study to report a negative vaccine effectiveness (-2 VE; 95% CI -182 to 63%) [29].

Fig. 3

Vaccine effectiveness against hospitalisation or hospitalisation and emergency department attendance for laboratory confirmed RVGE (only adjusted effect estimates included)

Pooled estimates for case-control studies in high income settings (89% VE; 95% CI 84-92%; 12 studies [13 estimates]) were comparable to the three unpooled cohort study estimates. All cohort studies were conducted in high income settings. One study reported an adjusted VE estimate of 87% (95% CI 80-92%) and other two studies stated unadjusted estimates of 100% (95% CI 87-100%), and 90% (95% CI 82-95%) [20, 23, 30].

Sensitivity analysis

We identified the possibility of study bias using the Newcastle-Ottawa Scale. The pooled OR for studies with a low risk of bias was 0.18 (95% CI 0.13-0.25; 13 studies [14 estimates], p<0.001), suggesting that any bias in these studies may have been minimal. The corresponding pooled estimates by World Bank classification did not change substantially. When studies from countries with state-based or private rotavirus vaccine provision (Castilla et al 2012, Martinon-Torres, 2011 and Muhsen et al., 2010) were dropped from the meta-analysis the pooled OR remained similar (0.19 OR; 95% CI 0.14-0.25; 18 studies [19 estimates], p<0.001) [24, 31, 42].

To further investigate the potential for publication bias, specifically that the effect of industry funding, we excluded 4 studies which had measured RVGE hospitalisations but were funded by industry [16, 32,33,34]. Overall pooled ORs remained comparable (0.19 OR; 95% CI 0.14-0.26 ; 17 studies [18 estimates], p<0.001). We also established that including an estimate from a study which measured vaccine effectiveness against severe disease (Vesikari score ≥11), did not cause an overestimation of the pooled estimate [16].

Subgroup analyses

Age

Because the age groups included in studies were varied we grouped ages in to the following groups to assess VE by crude age groups: <1 year, <2 years, >1 year and 1-2 years of age. In middle-income countries there was some variation in pooled estimates by age group but confidence intervals overlapped between estimates (Fig. 4a). Here estimates should be interpreted with caution as there was only 1 study in each of the 1-2 year and <2 year groups. Additionally, the VE estimate for children aged >12 months from the 2014 study by Cotes-Cantillo stood out as being heterogeneous. As the authors note this is likely to be due to the low sample size in this group and potentially due to a variation in strain dominance by age [29]. Estimates were very similar across age ranges in high-income countries (Fig. 4b).

Fig. 4

Vaccine effectiveness against hospitalisation or hospitalisation and emergency department attendance for laboratory confirmed RVGE comparing partial age groups a middle income countries, b high income countries. (only adjusted effect estimates included)

Full v partial vaccine dose

To determine the impact of the number of vaccine doses on vaccine effectiveness we compared studies which reported full dose vaccination with first dose vaccination for RV1 and RV5 vaccine effectiveness estimates. There were 21 studies which either reported full or partial vaccination giving 32 estimates (13 partial schedule; 19 full vaccine schedule). Pooled vaccine effectiveness for full dose (81% VE; 95% CI 75-86%, p<0.001) was higher than for partial dose (62% VE; 95% CI 55-69%, p<0.001). However, there was moderate to high heterogeneity for studies reporting estimates for full dose (I²= 60.2%, p<0.001). When stratified by World Bank classification, this difference was most pronounced in middle income countries, where pooled vaccine effectiveness for full dose was 74% (95% CI 67-80%, p<0.001) and 57% (95% CI 47-66%, p<0.001) for partial dose (Fig. 5a). Wider confidence intervals were reported in studies reporting full vaccination, likely due to smaller available sample sizes. In high income countries VE for partial vaccine dose was 72% (95% CI 60-80%, p<0.001) compared to 87% (95% CI 81-91%, p<0.001) for full dose (Fig. 5b).

Fig. 5

Vaccine effectiveness against hospitalisation or hospitalisation and emergency department attendance for laboratory confirmed RVGE comparing partial dose to full dose a middle income countries, b high income countries (only adjusted effect estimates included)

RV1 and RV5 vaccine effectiveness

Pooled VE for RV1 and RV5 stratified by World Bank country classification showed that RV5 has slightly higher VE point estimates in both high and middle income countries but this difference was not significant. RV1 is the predominant vaccine used in studies from middle income countries included in the meta-analysis (7/10). In high income countries three studies reported VE for RV1, six for RV5 and four RV1 and RV5 combined.

Meta-analysis of vaccine effectiveness against emergency department attendances for RVGE

There were 4 estimates from three studies that included measure of VE for ED attendances for RVGE [28, 29, 34]. Three estimates were from high income countries and one from middle-income. Publication bias was not assessed as there were inadequate numbers of studies to properly assess via a Begg’s. Heterogeneity was high for these studies (I2=78.7%, p<0.001). Random effects meta-analysis gave a pooled OR of 0.26 (95% CI 0.12-0.57, p=0.001), indicating a significant effect of vaccination against ED attendances for RVGE (Fig. 6). Analysis stratified by World Bank country classifications showed significant VE of 81% (95% CI 75-86%, p<0.001) for studies from high income countries. There was only one study from middle income countries with a VE of 16% (95% CI -79% to 61%, p=0.651).

Fig. 6

Vaccine effectiveness against emergency department attendances for RVGE (only adjusted effect estimates included)

Meta-analysis of vaccine effectiveness against community consultations for AGE

A single study reported full dose vaccine effectiveness against community consultations for laboratory confirmed RVGE, reporting a high unadjusted VE estimate of 96% (95% CI 76-100%) [20, 21]. However, four cohort studies included estimates of vaccine effectiveness for community consultations for AGE [17,18,19,20,21]. Three studies were from high income countries and one from middle-income. Only two out of the four studies reported adjusted estimates. The study by Nolan et al., 2012 only reported an adjusted VE for 1 or more doses [19]. We therefore could only include three cohort studies which report unadjusted VE in the meta-analysis. Publication bias was not assessed as there were inadequate numbers of studies to properly assess via a Begg’s test. Heterogeneity was very high for these studies (I²=97.9%, p<0.001). Random effects meta-analysis gave a pooled RR of 0.74 (95% CI 0.52-1.06, p=0.10), indicating a non-significant effect of vaccination against community consultations for AGE (Fig. 7). However, analysis stratified by World Bank country classifications showed significant VE of 40% (95% CI 13-58%, p=0.008) for studies from high income countries. There was only one study from middle income countries with a VE of -24% (95% -67% to 8%, p=0.157), this study was assessed as having a high risk of bias as crude VE was calculated by the authors and children in the vaccinated cohort were significantly younger than the unvaccinated cohort.

Fig. 7

Vaccine effectiveness against community consultations for AGE (only unadjusted effect estimates included)

Discussion

The pooled data from case-control studies indicates that RV vaccination is highly effective for preventing hospitalisations and / or ED attendances for laboratory confirmed rotavirus, as VE was 89% (95% CI 84-92) for high income countries and 74% (95% CI 67-80) for middle income countries. This finding is further supported by comparable estimates from the unpooled cohort studies. Most studies reporting VE for community consultations could not identify the causative organism and therefore reported a much lower effectiveness against community consultations for AGE (26% VE; 95% CI -6 to 48%). However, like VE against RVGE hospitalisations and ED attendances, the pooled VE for community consultations for AGE was significant in high income countries (40% VE; 95% CI 13-58%). This study was also able to show that studies from countries with similar economic classification demonstrate similar effectiveness and that effectiveness was lower in middle income countries compared with high income countries. Vaccine effectiveness against RVGE hospitalisations in high income countries (89% VE; 95% CI 84-92%) was consistent with but at the upper limit of that reported in reported in a Cochrane Review of RCTs; 85% (95% CI 80-88%) for RV1 and 82% (95% CI 50-93%) for RV5 for preventing severe rotavirus diarrhoea in children up to two years of age [6]. Differing study definitions of severity and age maybe responsible for the slight difference between pooled estimates. Additionally, pooled VE estimates from meta-analysis for RVGE hospitalisations in middle income countries (Brazil, Colombia, Nicaragua and Bolivia) were significantly lower than high-income countries, again consistent with estimates from efficacy studies [6, 48].

Our systematic review found substantial differences in the quality and design of studies and considerable heterogeneity. However, there was no evidence of publication bias. Heterogeneity was dealt with by stratifying the analysis by World Bank income classification when heterogeneity was low to moderate and by using sensitivity analysis to identify factors which may have caused bias in the overall estimate. We conclude that the best pooled estimates are provided by stratifying by World Bank country classification. Sensitivity analyses for RVGE hospitalisations and or ED attendances did not identify any substantial effects resulting from differences in study quality. Exclusion of studies from countries with only state based or private provision of RV vaccination made little difference to the overall effect estimates.

Sub-analyses by vaccine dose identified that 1 dose of RV vaccine conferred a lower overall VE estimate than full course dose, particularly in middle income countries, indicating that there is a clear benefit for children completing the full schedule. Wider confidence intervals were seen for partial dose estimates due to smaller available sample sizes which are likely a result of the majority of children completing recommended schedules.

It was difficult to assess VE by age as the different studies reported VE for different age groups and therefore finding standard categorisations for age was problematic. Pooled VE estimates in children aged >1 year of age in middle income countries were lower than for that for infants <1 year of age. This could be due to the higher relative disease burden in infants, a consequence of acquiring natural immunity with age, independent of vaccination [49]. Nonetheless, classifications used here showed no significant differences between age groups in both high and middle income countries, suggesting that the RV vaccination is highly effective against RVGE hospitalisation regardless of age. Interestingly, one study from Colombia reported high VE in the <1 year olds (84.4% VE; 95% CI 23-97%) but negative VE in >1 year olds (-79% VE; 95% CI -556 to 51%), the authors indicated that this could be because of a low sample size in the older group and a change in the predominant rotavirus strains circulating in Colombia during the study to heterotypic non-vaccine strains [29].

Additionally, whilst prior to vaccination the predominant strain type in many high income countries was G1P [8, 50] in lower and middle income countries there is often greater strain diversity and concurrent circulation of several strains simultaneously [1, 50]. It is possible that the frequency of more strains that heteroptypic to vaccine types may contribute to the lower effectiveness of the vaccine in middle income countries. However, there are likely to be inherent immununological and epidemiological factors at play [51].

Using meta-analysis to review VE against community consultations for AGE is particularly important in high income countries where the majority of the healthcare burden from rotavirus infection is in community healthcare settings. For instance in the UK rotavirus was deemed to be responsible for approximately 800 000 general practice consultations per year prior to vaccine introduction [4]. Pooled estimates presented here show a considerably lower vaccine effectiveness against AGE community consultations compared with RVGE hospitalisations in high income countries. Since only one study is available for middle income countries, evaluation of the effectiveness of vaccination against AGE community consultations in this setting is difficult, particularly because this study had high risk of bias predominantly due to significant differences in age of the vaccinated and unvaccinated cohorts.

Whereas clinical trials have suggested lower efficacy against milder disease [48, 52], a single study here reported VE against laboratory confirmed RVGE community consultations on par with VE against RVGE hospitalisations [20, 21]. Whilst there was no-indicator of disease severity in the study, the healthcare setting suggests milder disease. Therefore there is a need for more robustly designed studies in middle income settings and high income countries in order to properly assess RV vaccine effectiveness against milder disease resulting in community consultations for RVGE.

Studies reviewed here represent countries with a World Bank country classification of high or middle income, and clearly show lower VE in middle income countries. However, the burden of disease is likely to be greater in middle income countries representing a superior population greater potential benefit of vaccination in these settings. At time of review no studies could be included from low income settings However, findings from a recent study in Malawi suggest a VE of 64% (95% CI 24–83), similar to that reported in middle income countries [53]. Future reviews will be required to capture studies from these settings.

As these studies were conducted in the “real world” with population level vaccine introduction the VE estimates are likely to include both the direct effect and any herd protective effect of population vaccination. This could be particularly significant in cohort studies with high population vaccine uptake. Indeed only one study attempted to separate the herd protection and direct effect from the overall effect of vaccination, estimating a substantial increase in indirect effect of vaccination as cohort vaccine uptake increased over time [30]. More evidence of the indirect effect of vaccination is therefore required through subsequent cohort studies.

We searched three widely used databases—PubMed, Web of Science, and Academic Search Premier —as well as grey literature using a pre-specified, systematic search protocol. We were able to quality assess the studies included using an established critically appraised tool specifically for use with non-randomised studies in meta-analysis, allowing a good understanding of a studies validity importantly with reference to RCT as a gold standard. The majority of studies were assessed as being at low or moderate risk of bias strengthening the meta-analysis for RVGE hospitalisation. However, our assessment of study bias used one specific tool, the Newcastle-Ottawa Scale, with some author defined criteria, therefore it is possible that another bias assessment tool and criteria would identify different risk of bias. Furthermore, variations in study outcome definitions and statistical methods could have introduced error into some of our meta-analyses. For instance, heterogeneity was moderate to high in some of the meta-analyses, particularly those that examined community consultations; this could be related to varying definitions of a community consultation.

Conclusions

This review and meta-analysis has enabled the systematic production of pooled VE estimates for rotavirus vaccination globally from the literature. We conclude that RV vaccines represent a highly effective preventive measure against severe rotavirus disease, with “real world” vaccine effectiveness estimates as high as efficacy measures from RCTs. There is sufficient evidence to promote the continued roll out of both RV vaccines in both high and middle income settings. The modest benefits from vaccination against community consultations for RVGE represent information which can be used in appropriate cost-effectiveness studies, which may provide better understanding of the value of reducing mild to moderate disease through vaccination.