Skip to main content

A data-driven metapopulation model for the Belgian COVID-19 epidemic: assessing the impact of lockdown and exit strategies

Abstract

Background

In response to the ongoing COVID-19 pandemic, several countries adopted measures of social distancing to a different degree. For many countries, after successfully curbing the initial wave, lockdown measures were gradually lifted. In Belgium, such relief started on May 4th with phase 1, followed by several subsequent phases over the next few weeks.

Methods

We analysed the expected impact of relaxing stringent lockdown measures taken according to the phased Belgian exit strategy. We developed a stochastic, data-informed, meta-population model that accounts for mixing and mobility of the age-structured population of Belgium. The model is calibrated to daily hospitalization data and is able to reproduce the outbreak at the national level. We consider different scenarios for relieving the lockdown, quantified in terms of relative reductions in pre-pandemic social mixing and mobility. We validate our assumptions by making comparisons with social contact data collected during and after the lockdown.

Results

Our model is able to successfully describe the initial wave of COVID-19 in Belgium and identifies interactions during leisure/other activities as pivotal in the exit strategy. Indeed, we find a smaller impact of school re-openings as compared to restarting leisure activities and re-openings of work places. We also assess the impact of case isolation of new (suspected) infections, and find that it allows re-establishing relatively more social interactions while still ensuring epidemic control. Scenarios predicting a second wave of hospitalizations were not observed, suggesting that the per-contact probability of infection has changed with respect to the pre-lockdown period.

Conclusions

Contacts during leisure activities are found to be most influential, followed by professional contacts and school contacts, respectively, for an impending second wave of COVID-19. Regular re-assessment of social contacts in the population is therefore crucial to adjust to evolving behavioral changes that can affect epidemic diffusion.

Peer Review reports

Background

The COVID-19 pandemic has put a massive burden on modern society. While the global death toll of the virus has risen above 500,000 reported deaths on the 15th of July [1], several countries are evaluating strategies to cope with the virus on the medium to long term. As during the first wave of the COVID-19 pandemic neither a vaccine nor adequate therapeutic options were available, non-pharmaceutical interventions have been proven effective in reducing the pressure on healthcare systems [26]. After a massive implementation of lockdown measures, affecting as much as one third of the global world population [7], governments have eased some of the social distancing measures. After imposing a lockdown on March 14th [8], the Belgian government curtailed some of these measures with a plan for a gradual reopening over several weeks, starting from the 4th of May. The absence of substantial population immunity after this first wave of COVID-19 in Belgium [9] increases the risk of subsequent large-scale outbreaks when interventions are relaxed which could result, when not contained, in new COVID-19 waves with large numbers of new confirmed cases and hospitalized persons. In this context, data-driven models of disease spread can provide useful insights into the expected impact of easing non-pharmaceutical interventions [2, 6, 10]. Here we present a scenario analysis of possible re-opening strategies easing lockdown measures based on a data-driven metapopulation model for Belgium for COVID-19 [11]. We compare the expected epidemic trajectories and, at the same time, we validate the modelled scenarios with social contact data collected during and after lockdown. We aim to identify which intervention strategies have the largest potential impact on disease spread, based on the scarce data available during the early stage of the pandemic.

Methods

We constructed a meta-population model for COVID-19, in order to study the Belgian epidemic. The model reproduces the demography of children (0-18 years) and adults (19 years and above) in the 581 different Belgian municipalities [12]. Publicly available data [13, 14] from a social contact survey conducted in Flanders (Belgium) anno 2010-2011 is used to inform mixing patterns of the population [1517]. Mobility data retrieved from the Belgian census [18] is used to reconstruct mobility fluxes due to school attendance and work. A stochastic compartmental model is used to describe the spread of COVID-19 in the population within each patch of the system. The model is fitted to national hospitalization data [19].

Compartmental patch model

We use an extended SEIR stochastic compartmental model (Fig. 1) in which we distinguish pre-symptomatic (Ip), asymptomatic (Ia), and symptomatic (Ims and Iss) transmission by assuming different transmission rates, governed by different contact patterns during daytime and night-time as two time steps per simulated day. In particular, we assume that symptomatic individuals (both mildly symptomatic Ims and severely symptomatic Iss) reduce their number of contacts (following observations made during the 2009 Influenza pandemic [20]) and their commute (school/work) mobility. A fraction of symptomatic adults can show severe symptoms and thereforef require hospitalization (H) [5, 21]. Once this happens, we assume that they cannot further infect other people due to isolation measures [22]. We assume that children have a 50% lower susceptibility to infection compared to adults [4, 23, 24]. Table 1 shows a summary of the model parameters and the distributional assumptions thereabout.

Fig. 1
figure 1

Schematic representation of the compartmental model: Individuals start as susceptible (S) and can become exposed to the disease (E) when interacting with infected individuals (Ip,Ia,Ims and Iss). After a latent period, exposed individuals enter a pre-symptomatic phase (Ip), after which they can either become symptomatic (Ims and Iss) or remain without symptoms (Ia). Symptomatic individuals can develop mild symptoms (Ims) or severe symptoms (Iss). When symptoms are severe, they are hospitalized (H). The final outcome of infected individuals is either recovery (R) or death (D)

Table 1 Overview of the model parameters

Population mixing

According to the so-called social contact hypothesis [25] the number of contacts is proportional to the transmission probability of the disease, allowing to use empirically collected contact matrices to model disease transmission. Population mixing is informed by social contact data for different locations (work, home, school, transportation, leisure activity and other) during weekdays and weekends [15, 16], accessible through the Socrates tool [13, 14]. An asymptomatic individual interacts according to a contact matrix that is the sum of the contact matrices that correspond to different locations:

$$ \begin{aligned} C_{\text{asympt}}&=C_{\text{home}}+C_{\text{work}}+C_{\text{school}}+ C_{\text{leisure}}\\&\quad+ C_{\text{transport}}+ C_{\text{other}} \end{aligned} $$
(1)

The locations considered are the main ones used to classify contact location in social contact surveys [26, 27] and include an overall category for leisure activities (e.g. going to the gym, to a bar, etc.). Contributions from work, school and transport contacts are considered only during daytime timesteps of the simulation, whereas other contributions are considered both for daytime and night-time timesteps. Given the strong age-specific severity of COVID-19, we assume that, when symptomatic, adults reduce their contacts in a location-specific fashion, as reported during the 2009 H1N1 pandemic [20]:

$$ \begin{aligned} C_{\text{sympt}}&=C_{\text{home}}+0.09\cdot C_{\text{work}}+0.06\cdot C_{\text{leisure}}\\&\quad+0.13\cdot C_{\text{transport}}+ 0.25\cdot C_{\text{other}} \end{aligned} $$
(2)

We assume that children do not change behavior when symptomatic, as they are more likely to present fewer and milder symptoms as compared to adults [2830].

When intervention measures are implemented (see “Exit strategies” section), location-specific contacts are reduced. This has an impact on both Csympt and Casympt, implicitly assuming that a reduction in contacts because of symptoms is the same during the pre-pandemic and intervention period. The contact matrices then become:

$$ \begin{aligned} C_{\text{asympt}}&=C_{\text{home}}+p_{\mathrm{w}}\cdot C_{\text{work}}+p_{\mathrm{s}}\cdot C_{\text{school}}\\&\quad+p_{\mathrm{o}}\cdot C_{\text{leisure}}+p_{\mathrm{w}}\cdot C_{\text{transport}}+p_{\mathrm{o}}\cdot C_{\text{other}} \end{aligned} $$
(3)
$$ \begin{aligned} C_{\text{sympt}}\!&=\!C_{\text{home}}+p_{\mathrm{w}}\cdot 0.09\cdot C_{\text{work}}+ p_{\mathrm{o}}\cdot0.06\cdot C_{\text{leisure}}\\&\quad+p_{\mathrm{w}}\cdot 0.13\cdot C_{\text{transport}}+ p_{\mathrm{o}}\cdot0.25\cdot C_{\text{other}} \end{aligned} $$
(4)

where pw,ps,po are the percentages of contacts at work, at school and during leisure/other activities.

Contact matrices depend explicitly on the day of the week, as contact patterns during weekdays are profoundly different from contact patterns during the weekend.

Population mobility

Data from the Belgian census [18] is used to infer the daily commuting network among different Belgian municipalities. These mobility patterns capture the regular, day-to-day movement of individuals to reach their working/studying place. Commuting individuals make contacts in their residence municipality during the night and in their work/school municipality during the day. This is captured via the force of infection, described in detail in the next section.

More details on population mobility can be found in the Supporting Information. We assume that telework and school closure, in addition to reducing contacts at work and at school, reduce the mobility of the corresponding age class. So, for example, if teleworking is reducing contact at work by 60%, also the mobility of adults is reduced by 60%. Considered values of mobility reduction for adults (ma) and children (mc) for each scenario are reported in Table 2.

Table 2 Timing and concepts of lockdown relief

Force of infection

The force of infection for age class i and patch p is computed at each time step as:

$$\begin{array}{@{}rcl@{}} \lambda (i,p,t)&=\beta \sum_{j} \left[ \text{Susc}^{i} C^{\text{asympt}}_{i,j}(t) \text{Inf}^{\ j} \frac{I^{p}_{p,\ j}(t)+I^{p}_{a,\ j}(t)}{N^{p}(t)} \right. \\ &\qquad\left.+ \text{Susc}^{i} C^{\text{sympt}}_{i,j}(t)\text{Inf}^{\ j} \frac{I^{p}_{\text{ms},\ j}(t)+I^{p}_{\text{ss},\ j}(t)}{N^{p}(t)}\right] \end{array} $$
(5)

where \(I^{p}_{x\ j}(t)\) is the number of infectious individuals of infection class x belonging to age class j present in patch p at time t and Np(t) is the total patch population at time t. Equation (5) can account for different susceptibility (Susci) and infectivity (Infj) for age classes i and j. The contact matrices used depend on time because of week/weekend cycles and because of the intervention strategies implemented at any given time (see following sections). The force of infection presents an additional dependence on time, as depending on the time steps commuting individuals may contribute to the force of infection of their residence patch or of their destination patch. This contribution is considered both in the infected terms \(I^{p}_{x\ j}(t)\) as well as in the population term Np(t).

Interventions implemented on the 14th of March 2020

Starting from Friday 13th of March at midnight, Belgian authorities have declared the nation-wide closure of schools and universities, together with restaurants, cafes and gyms. Also, public gatherings were banished. On the 17th of March, further dispositions were put in place, limiting mobility of people in addition to closing companies and shops offering non-essential services. We model interventions by reducing mixing and mobility in the population (see “Population mixing” and “Population mobility” sections), with a compliance that increases linearly with time and reaches full compliance on the 23rd of March.

Calibration

To calibrate our model, we used publicly-available national data on daily hospital admissions [19]. We use Bayesian Optimization [31, 32] to maximize the likelihood of the simulated number of hospitalizations, given the observed data. We estimate the per-contact transmission probability (β), the number of initial infected, the reduction of the contact matrix during intervention with respect to the pre-pandemic period and time to reach full compliance (see Table 1). More details on the calibration procedure can be found in the Additional file 1.

Exit strategies

The Belgian government lifted the lockdown gradually from the 4th of May onward. Table 2 shows a simplified summary of the different phases and their implementation. Changes with respect to the previous phase (i.e. the previous row) are shown in bold. In our scenario analysis we considered three phases:

  • Phase 1: from the 4th of May, increasing the contacts made at work and during commuting by adults, to account for the increase of people going back to work. Mobility of adults increases accordingly.

  • Phase 2: from the 18th of May, increasing contacts made at school and during commuting by children to account for school re-opening. Mobility of children increases accordingly.

  • Phase 3: from the 8th of June, increasing contacts made during leisure and in other locations, to assess the impact of a possible re-opening of leisure activities.

For these phases we considered a compliance that increases linearly with time and reaches full compliance after one week.

Case isolation

When extensive contact tracing and testing is available, a viable option for disease mitigation is to isolate individuals that are infected. We assume that case isolation affects both symptomatic and asymptomatic individuals and we present our results in terms of a synthetic quantity, the parameter α, that is the percentage of individuals entering the symptomatic/asymptomatic class (Ia,Ims and Iss) that are effectively isolated. We assume that these isolated individuals reduce their contacts by a factor of ten. We do not cover here how to link the target α to an optimal strategy for contact tracing and testing. Such a strategy should take into account feasibility thereof in terms of the number of index cases that can be traced, test features (e.g. sensitivity, specificity), and willingness to report contacts [22, 33, 34]. We also assume that no isolation of pre-symptomatic people is implemented (Ip). We considered that case isolation can start at the beginning of phase 2 (i.e. on the 18th of May) or at the beginning of phase 3 (i.e. on the 8th of June), to assess the impact of delay in implementation.

Results

Impact of lockdown

Figure 2 shows the daily number of new hospitalizations in the initial phase of the epidemic, compared with our best model fit. Hospitalization data up to the 21st of March are consistent with an exponential growth model with a doubling time of 3.09 days (95% CI [3.05:3.11]) (red line). Combined with our estimated model parameters, this results in a basic reproduction number R0=3.40 (95% CI [3.36:3.44]). A strong, periodic effect on the reported number of hospital admissions can be observed, most likely due to delays in hospitalization during weekends. The no-intervention model is in line with hospitalization data up to the 21th of March, showing that interventions took about one week to impact hospitalizations. The model including the effect of interventions (green line) is compatible with an overall reduction in the total number of contacts of 85% with respect to the period prior the COVID-19 pandemic (see Additional file 1 for additional information on contact matrices).

Fig. 2
figure 2

Model fitting. Data on hospital admissions is shown in comparison with the best-fit model. Black points are used to calibrate the model in the lockdown phase. In both panels median curves are shown along with 50% confidence intervals (CIs; dark shade) and 95% CI (light shade)

Scenario analysis for lifting lockdown

Figure 3 shows the impact of the different phases of the exit strategy on the number of new hospitalizations, considering different implementations (i.e. parameter values) for each phase. We present estimation of the number of new hospitalizations up to the 31st of August, as considering a longer timeframe would require additional assumptions with regard to social distancing after the summer period. Results for the whole year are reported in the Additional file 1 (Figure S4). In Fig. 3a, at the beginning of phase 1 (4th of May), contacts at work and on transportation are increased, ranging from 10% to 40% of pre-pandemic values. As expected, there is a delay between the implementation of the first phase and its effect on the number of hospital admissions: after 3 weeks the number of hospital admissions stops decreasing as compared to the lockdown scenario. One further week is required to see differences between the three implementations of phase 1. In Fig. 3b we show the impact of phase 2 (school re-opening) once phase 1 is implemented for the smallest value of contacts at work/transportation considered (10%). The percentage of school contacts ranges from 10% to 40%. In this case, the different curves start to diverge 4 weeks after the re-opening of schools. Summer school holidays, starting on the 1st of July have a considerable (delayed) effect on the number of hospital admissions only in the 40% school contacts scenario. In Fig. 3c we show the impact of phase 3, once phase 1 and 2 are implemented with the smallest values of the considered parameters. Different implementations (i.e. parameter values) of phase 3 give different results after three weeks. Comparing the three panels, it is clear that changing the implementation of phase 3 has a larger impact than changing implementation of phase 1 or 2. The larger impact of phase 3 (leisure/other activities) is confirmed when comparing all the scenarios we considered.

Fig. 3
figure 3

Exit scenarios using different timings and location-specific reductions. a: different implementations of phase 1 (work re-opening). b: different implementations of phase 2 (school re-opening). c: different implementations of phase 3 (leisure re-opening). The top of each panel shows the parameter values used. In all panels median curves are shown along with 50% confidence intervals (dark shade) and 95% CI (light shade). Color-code is consistent across panels, with the same color marking the same scenario in different panels

Figure 4 shows the number of daily hospitalization and the cumulative number of hospitalizations up to the 31st of August (results up to the 31st of December are available in Additional file 1: Figure S4). Results are shown with respect to the scenario in which the lockdown would continue until the end of the simulation (green, solid line of panel a,b and c of Fig. 4) and that would result in 23.000 hospitalizations by the end of August. A smaller impact for school re-opening with respect to work and leisure re-opening is observed, both for peak hospitalizations and for total hospitalizations. Increasing the contacts at school by 10% (i.e. considering a different symbol marker but same color along the y-axis) has a smaller impact than increasing contacts at work (i.e. same symbol, different color along the y-axis) or leisure/other contacts (i.e. moving along the x-axis) of the same amount. Increasing contacts at work has a smaller impact in terms of peak hospitalizations than increasing leisure/other contacts; a similar impact is instead observed for the total number of hospitalizations. When considering results over the whole year (Additional file 1: Figure S4) the relative increase in the epidemic peak is weakly affected. The total final size, instead, increases for all scenarios, as the daily number of hospitalizations is summed up over a longer period of time.

Fig. 4
figure 4

Summary of exit scenarios. a: peak value of daily hospital admissions up to the 31st of August. b: number of hospitalizations up to the 31st of August. In both panels the y-axis shows the relative variation with respect to the best-case (least contacts) scenario. A circle denotes the scenario used in the contact isolation analysis (Fig. 6)

However, hospitalization data is compatible with the lockdown scenario (Fig. 2) up to the end of June. Comparison of the contact matrices used in the model with the results of a recent social contact survey targeting Belgian adults during and after the lockdown provides a means to interpret this. Figure 5 shows the measured contact matrices in comparison to the ones of the simulated scenarios. For the empirical contact matrices we report the average number of contacts, together with bootstrap confidence intervals (n=10,000), whereas for the ones of the model we report the average number of contacts and the min/max values considered in the scenarios listed in Table 2 and shown in Fig. 3. Our model uses a higher number of contacts during phase 1 with respect to empirical data, whereas for phase 2 and 3 we observe overlapping intervals for the number of contacts used in the model and measured by the empirical data. Data for children is however not available (hence marked with an “X” in Fig. 5), making a full comparison with empirical data not possible.

Fig. 5
figure 5

Comparison of model contact matrix and measured ones. a-c: Contact matrices for phase 1 (a), phase 2 (b) and phase 3 (c) in the simulated scenarios. For each matrix element we report the average value and the [min:max] interval over the different implementations of phases 1, phase 2 and phase 3 considered in Fig. 5. d-f: Contact matrices for phase 1 (d), phase 2 (e) and phase 3 (f) measured in a survey representative of the Belgian adult population. For each matrix element we report the average value and the 95% bootstrap confidence interval. Contacts of children participants, not measured in the survey, are marked with “X”. Data from [50], available at [59]

Case isolation

Figure 6 shows the impact of case isolation on the scenario marked with a circle in Fig. 4 (10% contacts at work/transportation, 40% contacts at school and 30% leisure/other contacts scenario marked with a circle in Fig. 4). The ability to isolate newly infected individuals has a considerable impact on the number of hospital admissions. The isolation of 25% of new cases is able to reduce the expected number of hospital admission at the end of August by 25%. The isolation of twice as many cases (50% instead of 25%) would lead to a reduction of 37% of admissions. Starting case isolation 3 weeks after (at the start of phase 3 instead of phase 2) lessens the reduction to 21% from 25%. A stronger effect of this delay is measured in the 50% case isolation scenario: in this case, starting the isolation at the start of phase 3 decreases the reduction in admissions from 37% to 28%.

Fig. 6
figure 6

Effect of case isolation in a specific scenario. a: new hospitalizations per day. b: cumulative number of hospitalizations relative to the no case isolation scenario. All curves are obtained considering 40% of working contacts, 40% of contacts at school and 40% of leisure/other contacts with respect to pre-pandemic period (scenario denoted by a black circle in Fig. 4). In both panels median curves are shown along with 50% confidence intervals (dark shade) and 95% CI (light shade)

Discussion

We used a stochastic, discrete time, data-driven meta-population model to evaluate several scenarios for lifting the lockdown in three phases. The model includes data on pre-pandemic mobility and mixing, and is calibrated on hospital admissions. The initial phase of the COVID-19 epidemic in Belgium is characterized by a fast spread of the disease, with a doubling time of 3.09 days (95% CI [3.05:3.14]), in line with values from other countries [2, 5, 3537]. Combined with our parameter choices, this results in R0=3.40 (95% CI [3.36:3.44]), which lies within the interval estimated in recent meta-analysis (mean =2.6, standard deviation =0.54 [38] and mean =3.28 [37]). Our model appropriately describes hospital admissions during the lockdown period if a strong reduction (85%) in the number of contacts is established. In this situation the number of hospital admissions starts to decrease 3 weeks after the start of the lockdown allowing the healthcare system to cope with ICU demands. At the end of the lockdown, the reproduction number is estimated to be 0.73 (95% CI [0.70:0.76]). Such a strong reduction in the average number of contacts marks the disruption that a lockdown has on everyday life. Studies in Wuhan and Shanghai [4] found an even stronger reduction in the number of contacts during lockdown, while a recent survey in the UK [38] measured a reduction of 75%. Preliminary analysis of social contact data collected in Belgium after the lockdown shows similar results as compared to [4] and [38], in line with our modelling results. Adherence to country-specific contact data is paramount, as intervention measures can vary substantially between countries, both in terms of implementation and in terms of compliance. Collecting country specific contact data during the different stages of the epidemic (i.e. before, during and after intervention) is therefore of crucial importance to adequately assess the impact of social distancing. Nevertheless, our knowledge of contact patterns before the COVID-19 crisis can be used to identify the relative impact of introducing social distancing in different locations. As we expect different locations to contribute to COVID-19 diffusion according to their location specific contact patterns, we can assess the impact of intervention strategies formalizing them in location specific reductions of contacts. In the current analysis this approach was taken, whilst considering a plausible range of reductions in social contacts in different circumstances. According to our model, leisure activities have the largest potential impact on the epidemic profile. This is consistent with leisure/other contacts accounting for 25% to 40% of the total contacts people make, according to representative surveys [26, 27]. However, the absence of a resurgence of hospitalizations by the end of June suggests that there is a smaller per-contact probability of transmission after lockdown with respect to pre-lockdown. This could be due to behavioral changes in how contacts are established (i.e. increased inter-personal distance or the wearing of face masks [39]) after the lockdown or to environmental factors (e.g. humidity and temperature [40]) that could affect transmission. As a matter of fact, surveys in Belgium [41] have documented a marked increase in outdoor contacts and face-mask wearing during the three phases of lockdown relief (Figure S3 in Additional file 1), supporting this hypothesis. In the light of that, our result are useful in establishing a hierarchy of location-specific contacts, but a careful interpretation of the absolute number of infections is necessary.

We observed less impact of school closure on hospital admissions in contrast to social mixing at work and during transport or leisure activities. First, as expected, school closure leads to observable effects only in those scenarios in which a consistent fraction (i.e. 40% or more) of school contacts are established in the population. Second, as children have a much lower probability of being symptomatic (and as such of being hospitalized) with respect to adults [37], increased diffusion among children increases the observed hospital admissions mostly indirectly, through the increase of infected adults. We tested, as a sensitivity analysis, a scenario in which children have the same susceptibility to the disease: in this case school closure would have a larger impact on the number of infections, especially in the children’s age class. The role of children is still unclear and, although their secondary attack rate in household is similar to the one of adults [29], there is evidence that they present smaller viral load [4245] and reduced transmissibility [28, 46], together with a lower number of confirmed cases with respect to adults [23]. This increased susceptibility scenario is therefore unlikely, given the information on COVID-19 we have so far.

Since the expected resurgence in the number of hospitalizations is not observed, this suggests that the proportionality factor between conversational contacts and transmission rates postulated in the so-called social contact hypothesis [25] has changed from the lockdown to the post-lockdown period. This is likely due to behavioral changes (increased hygiene, prominence of outdoor over indoor community contacts, face-mask wearing, etc.) reducing the per-average contact transmission probability. For instance, surveys [41] in Belgium have documented a marked increase in outdoor contacts and face-mask wearing during the three phases of lockdown relief (Figure S3 in Additional file 1), supporting this hypothesis. In our results, isolation of newly infected individuals has an important impact on epidemic mitigation. Implementing case isolation would allow to re-establish social interactions while still ensuring epidemic containment. We stress here that although we quantified the reduction of spreading potential in terms of number of contacts, this may also come as a combination of different effects, for example when antivirals to be used in the early phase of the infection will become available [47]. Also, a fast setup is crucial: a 3 weeks delay in implementing case isolation leads to a considerable impact on the number of new hospital admissions. As a fast and reliable contact tracing is of foremost importance, several digital solutions have been proposed to match the need for personal information with privacy concerns [48, 49].

Our model assumptions result in a set of limitations. First, considering only two age classes does not allow to fully capture the heterogeneity involved in COVID-19 transmission, like increased burden on senior population. Including more age classes, however, would require assumptions for those age-specific parameters that have not been estimated for Belgium. In this sense our simplifying assumption, although less flexible, presents an easier to interpret picture.

The change in behavior for symptomatic individuals, and the corresponding reduction in the number of contacts, is informed by data collected during the 2009 H1N1 pandemic in the UK [20]. Although the COVID-19 pandemic presents different features with respect to the 2009 pandemic, a similar change of behavior in symptomatic individuals is expected. In particular, data collected in Belgium during and after the lockdown [50] has found that self-isolation when symptomatic is regarded as highly effective. Finally, although our model is specified at the municipality level, hospitalization data at the municipality level was not available at the time of conceiving of this study. We plan to expand our analysis at a smaller geographical scale in the future, to fully address heterogeneity in spatial transmission.

Other models have been applied to the emergence of COVID-19 in Belgium, either specifically [5155] or in multi-country applications [56]. Using different model paradigms allows to focus on distinct aspects of the outbreak, like delay distributions of the clinical history of patients [53], a more detailed and age-specific handling of serological data with MCMC [52] or exploring individual-specific heterogeneities in transmissions and contact tracing options [51]. When evaluating intervention strategies with profound societal impact, ideally different models should be compared [57, 58].

Conclusion

In conclusion, we show the predicted impact of a phase-based relief of lockdown measures taken in Belgium. Through validation using empirical data on social contacts and the observed trajectory of the epidemic, our results suggest that the per-contact probability of infection has changed from pre- to post-lockdown. While economic and societal needs urge governments to relieve strict distancing measures and mobility restrictions, caution is required. Contacts during leisure activities were found to be most influential, followed by professional contacts and school contacts, respectively, for an impending second wave of COVID-19. Regular re-assessment is crucial to adjust to evolving behavioral changes that can affect epidemic diffusion. In addition to social distancing, sufficient capacity for extensive testing and contact tracing is essential for successful mitigation.

Availability of data and materials

Demographic data is publicly available from Belgian Statistics. Surveillance data is publicly available and provided by the Belgian Scientific Institute for Public Health, Sciensano. Contact data are publicly available.

Declarations

References

  1. Dong E, Du H, Gardner L. An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect Dis. 2020; 20(5):533–4. https://doi.org/10.1016/S1473-3099(20)30120-1.

    Article  CAS  Google Scholar 

  2. Di Domenico L, Pullano G, Sabbatini CE, Boëlle P-Y, Colizza V. Impact of lockdown on COVID-19 epidemic in île-de-france and possible exit strategies. BMC Med. 2020; 18(1):240. https://doi.org/10.1186/s12916-020-01698-4.

    Article  CAS  Google Scholar 

  3. Prem K, Liu Y, Russell TW, Kucharski AJ, Eggo RM, Davies N, Flasche S, Clifford S, Pearson CAB, Munday JD, Abbott S, Gibbs H, Rosello A, Quilty BJ, Jombart T, Sun F, Diamond C, Gimma A, van Zandvoort K, Funk S, Jarvis CI, Edmunds WJ, Bosse NI, Hellewell J, Jit M, Klepac P. The effect of control strategies to reduce social mixing on outcomes of the COVID-19 epidemic in Wuhan, China: a modelling study. Lancet Publ Health. 2020:261–70. https://doi.org/10.1016/S2468-2667(20)30073-.

  4. Zhang J, Litvinova M, Liang Y, Wang Y, Wang W, Zhao S, Wu Q, Merler S, Viboud C, Vespignani A, Ajelli M, Yu H. Changes in contact patterns shape the dynamics of the COVID-19 outbreak in China. Science. 2020. https://doi.org/10.1126/science.abb8001.

  5. Riccardo F, Ajelli M, Andrianou X, Bella A, Del Manso M, Fabiani M, Bellino S, Boros S, Mateo Urdiales A, Marziano V, Rota MC, Filia A, D extquoterightAncona FP, Siddu A, Punzo O, Trentini F, Guzzetta G, Poletti P, Stefanelli P, Castrucci MR, Ciervo A, Di Benedetto C, Tallon M, Piccioli A, Brusaferro S, Rezza G, Merler S, Pezzotti P. Epidemiological characteristics of COVID-19 cases in italy and estimates of the reproductive numbers one month into the epidemic. medRxiv. 2020. https://doi.org/10.1101/2020.04.08.20056861.

  6. Ferguson NM, Laydon D, Nedjati-Gilani G, et al.Impact of non-pharmaceutical interventions (npis) to reduce COVID-19 mortality and healthcare demand. 2020. https://doi.org/10.25561/77482.

  7. Coronavirus: The world in lockdown in maps and charts. 2020. https://www.bbc.com/news/world-52103747. Accessed 24 Apr 2021.

  8. Data on country response measures to COVID-19 (ECDC). 2020. https://www.ecdc.europa.eu/en/publications-data/download-data-response-measures-covid-19. Accessed 24 Apr 2021.

  9. Herzog SA, De Bie J, Abrams S, Wouters I, Ekinci E, Patteet L, Coppens A, De Spiegeleer S, Beutels P, Van Damme P, Hens N, Theeten H. Seroprevalence of igg antibodies against sars coronavirus 2 in Belgium - a serial prospective cross-sectional nationwide study of residual samples. medRxiv. 2021. https://doi.org/10.1101/2020.06.08.20125179.

  10. Aleta A, Martin-Corral D, Pastore y Piontti A, Ajelli M, Litvinova M, Chinazzi M, Dean NE, Halloran ME, Longini IM, Merler S, Pentland A, Vespignani A, Moro E, Moreno Y. Modeling the impact of social distancing, testing, contact tracing and household quarantine on second-wave scenarios of the COVID-19 epidemic. medRxiv. 2020. https://doi.org/10.1101/2020.05.06.20092841.

  11. Coletti P, Libin P, Petrof O, Abrams S, Faes C, Kuylen E, Wambua J, Willem L, the SIMID COVID-19 team, Beutels P, Hens N. covid-19 report on a meta-population model for Belgium: a firststatus report. 2020. https://www.uhasselt.be/Images/DSI/report_meta.pdf/. Accessed 24 Apr 2021.

  12. Luca GD, Kerckhove KV, Coletti P, Poletto C, Bossuyt N, Hens N, Colizza V. The impact of regular school closure on seasonal influenza epidemics: a data-driven spatial transmission model for Belgium. BMC Infect Dis. 2018; 18(1):1–16. https://doi.org/10.1186/s12879-017-2934-3.

    Article  Google Scholar 

  13. Willem L, Hoang VT, Funk S, Coletti P, Beutels P, Hens N. SOCRATES: an online tool leveraging a social contact data sharing initiative to assess mitigation strategies for COVID-19. BMC Res Notes. 2020; 13(1):293. https://doi.org/10.1186/s13104-020-05136-9.

    Article  CAS  Google Scholar 

  14. SocRates Tool. 2020. http://www.socialcontactdata.org/socrates/. Accessed 24 Apr 2021.

  15. Kifle YW, Goeyvaerts N, Van Kerckhove K, Willem L, Kucharski A, Faes C, Leirs H, Hens N, Beutels P. Animal ownership and touching enrich the context of social contacts relevant to the spread of human infectious diseases. PLOS ONE. 2015; 10(7):1–13. https://doi.org/10.1371/journal.pone.0133461.

    Article  Google Scholar 

  16. Willem L, Van Kerckhove K, Chao DL, Hens N, Beutels P. A nice day for an infection? weather conditions and social contact patterns relevant to influenza transmission. PLOS ONE. 2012; 7(11):1–7. https://doi.org/10.1371/journal.pone.0048695.

    Article  Google Scholar 

  17. Hoang TV, Coletti P, Kifle YW, Kerckhove KV, Vercruysse S, Willem L, Beutels P, Hens N. Close contact infection dynamics over time: insights from a second large-scale social contact survey in flanders, Belgium, in 2010-2011. BMC Infect Dis. 2021; 21(1):274. https://doi.org/10.1186/s12879-021-05949-4.

    Article  Google Scholar 

  18. StatBel, the Belgian statistical office. 2020. https://statbel.fgov.be/en. Accessed 24 Apr 2021.

  19. Sciensano, Covid-19 Belgium Epidemiological Situation. 2020. https://datastudio.google.com/embed/u/0/reporting/c14a5cfc-cab7-4812-848c-0369173148ab/page/tpRKB. Accessed 24 Apr 2021.

  20. Van Kerckhove K, Hens N, Edmunds WJ, Eames KTD. The Impact of Illness on Social Networks: Implications for Transmission and Control of Influenza. Am J Epidemiol. 2013; 178(11):1655–62. https://doi.org/10.1093/aje/kwt196.

    Article  Google Scholar 

  21. Guan W-J, Ni Z-Y, Hu Y, Liang W-H, Ou C-Q, He J-X, Liu L, Shan H, Lei C-L, Hui DSC, Du B, Li L-J, Zeng G, Yuen K-Y, Chen R-C, Tang C-L, Wang T, Chen P-Y, Xiang J, Li S-Y, Wang J-L, Liang Z-J, Peng Y-X, Wei L, Liu Y, Hu Y-H, Peng P, Wang J-M, Liu J-Y, Chen Z, Li G, Zheng Z-J, Qiu S-Q, Luo J, Ye C-J, Zhu S-Y, Zhong N-S. Clinical characteristics of coronavirus disease 2019 in China. N Engl J Med. 2020; 382(18):1708–20. https://doi.org/10.1056/NEJMoa2002032.

    Article  CAS  Google Scholar 

  22. Hellewell J, Abbott S, Gimma A, Bosse NI, Jarvis CI, Russell TW, Munday JD, Kucharski AJ, Edmunds WJ, Sun F, Flasche S, Quilty BJ, Davies N, Liu Y, Clifford S, Klepac P, Jit M, Diamond C, Gibbs H, van Zandvoort K, Funk S, Eggo RM. Feasibility of controlling COVID-19 outbreaks by isolation of cases and contacts. Lancet Glob Health. 2020; 8(4):488–96. https://doi.org/10.1016/S2214-109X(20)30074-7.

    Article  Google Scholar 

  23. Ludvigsson JF. Systematic review of covid-19 in children shows milder cases and a better prognosis than adults. Acta Paediatr. 2020; 109(6):1088–95. https://doi.org/10.1111/apa.15270.

    Article  CAS  Google Scholar 

  24. Mizumoto K, Omori R, Nishiura H. Age specificity of cases and attack rate of novel coronavirus disease (COVID-19). medRxiv. 2020. https://doi.org/10.1101/2020.03.09.20033142.

  25. Wallinga J, Teunis P, Kretzschmar M. Using data on social contacts to estimate age-specific transmission parameters for respiratory-spread infectious agents. Am J Epidemiol. 2006; 164(10):936–44.

    Article  Google Scholar 

  26. Hoang T, Coletti P, Melegaro A, Wallinga J, Grijalva CG, Edmunds JW, Beutels P, Hens N. A Systematic Review of Social Contact Surveys to Inform Transmission Models of Close-contact Infections. Epidemiology. 2019; 30(5):723–36. https://doi.org/10.1097/EDE.0000000000001047.

    Article  Google Scholar 

  27. Mossong J, Hens N, Jit M, Beutels P, Auranen K, Mikolajczyk R, Massari M, Salmaso S, Tomba GS, Wallinga J, Heijne J, Sadkowska-Todys M, Rosinska M, Edmunds WJ. Social contacts and mixing patterns relevant to the spread of infectious diseases. PLOS Med. 2008; 5(3):1–1. https://doi.org/10.1371/journal.pmed.0050074.

    Article  Google Scholar 

  28. Boast A, Munro A, Goldstein H. An evidence summary of paediatric COVID-19 literature. 2020. https://doi.org/10.31440/DFTB.24063.

  29. Bi Q, Wu Y, Mei S, Ye C, Zou X, Zhang Z, Liu X, Wei L, Truelove SA, Zhang T, Gao W, Cheng C, Tang X, Wu X, Wu Y, Sun B, Huang S, Sun Y, Zhang J, Ma T, Lessler J, Feng T. Epidemiology and transmission of COVID-19 in 391 cases and 1286 of their close contacts in Shenzhen, China: a retrospective cohort study. Lancet Infect Dis. 2020; 3099(20):1–9. https://doi.org/10.1016/S1473-3099(20)30287-5.

    Google Scholar 

  30. Zimmermann P, Curtis N. Coronavirus infections in children including COVID-19: An overview of the epidemiology, clinical features, diagnosis, treatment and prevention options in children. Pediatr Infect Dis J. 2020; 39(5):355–68. https://doi.org/10.1097/INF.0000000000002660.

    Article  Google Scholar 

  31. Shahriari B, Swersky K, Wang Z, Adams RP, de Freitas N. Taking the human out of the loop: A review of bayesian optimization. Proc IEEE. 2016; 104(1):148–75. https://doi.org/10.1109/JPROC.2015.2494218.

    Article  Google Scholar 

  32. Jiménez J, Ginebra J. pygpgo: Bayesian optimization for python. J Open Source Softw. 2017; 2(19):431. https://doi.org/10.21105/joss.00431.

    Article  Google Scholar 

  33. Kucirka LM, Lauer SA, Laeyendecker O, Boon D, Lessler J. Variation in false-negative rate of reverse transcriptase polymerase chain reaction–based sars-cov-2 tests by time since exposure. Ann Intern Med. https://doi.org/10.7326/M20-1495.

  34. Kucharski AJ, Klepac P, Conlan A, Kissler SM, Tang M, Fry H, Gog J, Edmunds J. Effectiveness of isolation, testing, contact tracing and physical distancing on reducing transmission of sars-cov-2 in different settings. medRxiv. 2020. https://doi.org/10.1101/2020.04.23.20077024.

  35. Jit M, Jombart T, Nightingale ES, Endo A, Abbott S, LSHTM Centre for Mathematical Modelling of Infectious Diseases COVID-19 Working Group, Edmunds WJ. Estimating number of cases and spread of coronavirus disease (COVID-19) using critical care admissions, united kingdom, february to march 2020. Eurosurveillance. 2020; 25(18). https://doi.org/10.2807/1560-7917.ES.2020.25.18.2000632.

  36. Zhao S, Lin Q, Ran J, Musa SS, Yang G, Wang W, Lou Y, Gao D, Yang L, He D, Wang MH. Preliminary estimation of the basic reproduction number of novel coronavirus (2019-ncov) in China, from 2019 to 2020: A data-driven analysis in the early phase of the outbreak. Int J Infect Dis. 2020; 92:214–7. https://doi.org/10.1016/j.ijid.2020.01.050.

    Article  CAS  Google Scholar 

  37. Liu Y, Gayle AA, Wilder-Smith A, Rocklöv J. The reproductive number of COVID-19 is higher compared to SARS coronavirus. J Travel Med. 2020; 27(2). https://doi.org/10.1093/jtm/taaa021.

  38. Jarvis CI, Zandvoort KV, Gimma A, Prem K, working Group CC, Klepac P, Rubin GJ, Edmunds WJ. Quantifying the impact of physical distance measures on the transmission of COVID-19 in the UK. BMC Med. 2020. https://doi.org/10.1186/s12916-020-01597-8.

  39. Prather KA, Wang CC, Schooley RT. Reducing transmission of sars-cov-2. Science. 2020; 368(6498):1422–4. https://doi.org/10.1126/science.abc6197.

    Article  CAS  Google Scholar 

  40. Ozyigit A. Understanding COVID-19 transmission: The effect of temperature and health behavior on transmission rates. Infect Dis Health. 2020. https://doi.org/10.1016/j.idh.2020.07.001.

  41. The Grote Corona Studie. https://corona-studie.shinyapps.io/corona-studie/.

  42. Jones TC, Mü hlemann B, Veith T, Zuchowski M, Hofmann J, Stein A, Edelmann A, Corman VM, Drosten C. An analysis of sars-cov-2 viral load by patient age. 2020. https://zoonosen.charite.de/fileadmin/user_upload/microsites/m_cc05/virologie-ccm/dateien_upload/Weitere_Dateien/analysis-of-SARS-CoV-2-viral-load-by-patient-age.pdf. Accessed 24 Apr 2021.

  43. Held L. A discussion and reanalysis of the results reported in Jones et al (2020): "An analysis of SARS-CoV-2 viral load by patient age". 2020. https://osf.io/bkuar/. Accessed 24 Apr 2021.

  44. Stoye J. A critical assessment of some recent work on COVID-19. 2020. http://arxiv.org/abs/2005.10237. Accessed 24 Apr 2021.

  45. Curtis D. Children have lower sars-cov-2 viral loads than adults. Preprints. 2020. https://doi.org/10.20944/preprints202005.0367.v1.

  46. Heavey L, Casey G, Kelly C, Kelly D, McDarby G. No evidence of secondary transmission of covid-19 from children attending school in ireland, 2020. Eurosurveillance. 2020; 25(21). https://doi.org/10.2807/1560-7917.ES.2020.25.21.2000903.

  47. Torneri A, Libin P, Vanderlocht J, Vandamme A-M, Neyts J, Hens N. A prospect on the use of antiviral drugs to control local outbreaks of COVID-19. BMC Med. 2020; 18(1):191. https://doi.org/10.1186/s12916-020-01636-4.

    Article  CAS  Google Scholar 

  48. Ferretti L, Wymant C, Kendall M, Zhao L, Nurtay A, Abeler-Dörner L, Parker M, Bonsall D, Fraser C. Quantifying sars-cov-2 transmission suggests epidemic control with digital contact tracing. Science. 2020; 368(6491). https://doi.org/10.1126/science.abb6936.

  49. Ienca M, Vayena E. On the responsible use of digital data to tackle the COVID-19 pandemic Large-scale. Nat Med. 2020; 26(4):458. https://doi.org/10.1038/s41591-020-0823-6.

    Article  Google Scholar 

  50. Coletti P, Wambua J, Gimma A, Willem L, Vercruysse S, Vanhoutte B, Jarvis CI, Van Zandvoort K, Edmunds J, Beutels P, Hens N. Comix: comparing mixing patterns in the belgian population during and after lockdown. Sci Rep. 2020; 10(1):21885. https://doi.org/10.1038/s41598-020-78540-7.

    Article  CAS  Google Scholar 

  51. Willem L, Abrams S, Libin PJK, Coletti P, Kuylen E, Petrof O, Møgelmose S, Wambua J, Herzog SA, Faes C, Beutels P, Hens N. The impact of contact tracing and household bubbles on deconfinement strategies for COVID-19. Nat Commun. 2021; 12(1):1524. https://doi.org/10.1038/s41467-021-21747-7.

    Article  CAS  Google Scholar 

  52. Abrams S, Wambua J, Santermans E, Willem L, Kuylen E, Coletti P, Libin P, Faes C, Petrof O, Herzog SA, Beutels P, Hens N. Modeling the early phase of the Belgium COVID-19 epidemic using a stochastic compartmental model and studying its implied future trajectories. Epidemics. 2021:100449. https://doi.org/10.1016/j.epidem.2021.100449.

  53. Faes C, Abrams S, Van Beckhoven D, Meyfroidt G, Vlieghe E, Hens N, Belgian Collaborative, Group on COVID-19 Hospital Surveillance. Time between Symptom Onset, Hospitalisation and Recovery or Death: Statistical Analysis of Belgian COVID-19 Patients. Int J Environ Res Public Health. 2020; 17:7560. https://doi.org/10.3390/ijerph17207560.

    Article  CAS  Google Scholar 

  54. Libin PJK, Willem L, Verstraeten T, Torneri A, Vanderlocht J, Hens N. Assessing the feasibility and effectiveness of household-pooled universal testing to control COVID-19 epidemics. PLOS Comput Biol. 2021; 17(3):1–22. https://doi.org/10.1371/journal.pcbi.1008688.

    Article  Google Scholar 

  55. Franco N. Covid-19 Belgium: Extended seir-qd model with nursing homes and long-term scenarios-based forecasts. medRxiv. 2020. https://doi.org/10.1101/2020.09.07.20190108.

  56. Russell T, Hellewell J, Abbott S, Golding N, Gibbs H, Jarvis C, van Zandvoort K, CMMID nCov working group, Flasche S, Eggo R, JW E, Kucharski A. Using a delay-adjusted case fatality ratio to estimate under-reporting. 2020. https://cmmid.github.io/topics/covid19/global_cfr_estimates.html. Accessed 24 Apr 2021.

  57. Holmdahl I, Buckee C. Wrong but useful — what COVID-19 epidemiologic models can and cannot tell us. N Engl J Med. https://doi.org/10.1056/NEJMp2016822.

  58. Den Boon S, Jit M, Brisson M, Medley G, Beutels P, White R, Flasche S, Hollingsworth TD, Garske T, Pitzer VE, Hoogendoorn M, Geffen O, Clark A, Kim J, Hutubessy R. Guidelines for multi-model comparisons of the impact of infectious disease interventions. BMC Med. 2019; 17(1):1–13. https://doi.org/10.1186/s12916-019-1403-9.

    Article  CAS  Google Scholar 

  59. CoMix data. https://doi.org/10.5281/zenodo.4147585.

  60. Lauer SA, Grantz KH, Bi Q, Jones FK, Zheng Q, Meredith HR, Azman AS, Reich NG, Lessler J. The Incubation Period of Coronavirus Disease 2019 (COVID-19) From Publicly Reported Confirmed Cases: Estimation and Application. Ann Intern Med. 2020. https://doi.org/10.7326/M20-0504.

  61. WHO question & answer. 2020. https://www.who.int/news-room/q-a-detail/q-a-similarities-and-differences-covid-19-and-influenza. Accessed 24 Apr 2021.

  62. Day M. Covid-19: identifying and isolating asymptomatic people helped eliminate virus in italian village. BMJ. 2020; 368. https://doi.org/10.1136/bmj.m1165.

  63. Nishiura H, Kobayashi T, Suzuki A, Jung SM, Hayashi K, Kinoshita R, Yang Y, Yuan B, Akhmetzhanov AR, Linton NM, Miyama T. Estimation of the asymptomatic ratio of novel coronavirus infections (COVID-19). Int J Infect Dis IJID Off Publ Int Soc Infect Dis. 2020; 94:154–5. https://doi.org/10.1016/j.ijid.2020.03.020.

    CAS  Google Scholar 

Download references

Acknowledgements

We thank several researchers from the SIMID COVID-19 consortium from the University of Antwerp and Hasselt University for numerous constructive discussions and meetings. We thank Giulia Pullano, Laura Di Domenico and Vittoria Colizza for useful discussions. The authors are also very grateful for access to the data from the Belgian Scientific Institute for Public Health, Sciensano.

Funding

This work received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (PC and NH, grant number 682540 – TransMID project, PL, NH, PB grant number 101003688 – EpiPose project). SA and NH gratefully acknowledge support from the Fonds voor Wetenschappelijk Onderzoek (FWO) (RESTORE project – G0G2920N). LW received funding from the Research Foundation Flanders (1234620N). PL received funding from the Research Foundation Flanders (post-doctoral grant 1242021N). The resources and services used in this work were provided by the VSC (Flemish Supercomputer Center), funded by the Research Foundation – Flanders (FWO) and the Flemish Government.

Author information

Authors and Affiliations

Authors

Contributions

PC, PB and NH conceived the study. PC and PL contributed to the software development. PC, PL, SA, LW, PB and NH prepared the first draft of the manuscript. PC, SA, CF, OP, SAH, EK and JW contributed to the data preparation and/or collection. All authors contributed to the final version of the paper and approved the final version of the manuscript.

Each member of the SIMID COVID-19 team contributed in processing, cleaning and interpretation of data, interpreting findings, contributed to the manuscript, and approved the work for publication.

Corresponding author

Correspondence to Pietro Coletti.

Ethics declarations

Ethics approval and consent to participate

The data used in this study is publicly available in anonymised form.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1

Supporting information.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Coletti, P., Libin, P., Petrof, O. et al. A data-driven metapopulation model for the Belgian COVID-19 epidemic: assessing the impact of lockdown and exit strategies. BMC Infect Dis 21, 503 (2021). https://doi.org/10.1186/s12879-021-06092-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12879-021-06092-w

Keywords