Methods to discriminate primary from secondary dengue during acute symptomatic infection

Abstract

Background

Dengue virus infection results in a broad spectrum of clinical outcomes, ranging from asymptomatic infection through to severe dengue. Although prior infection with another viral serotype, i.e. secondary dengue, is known to be an important factor influencing disease severity, current methods to determine primary versus secondary immune status during the acute illness do not consider the rapidly evolving immune response, and their accuracy has rarely been evaluated against an independent gold standard.

Methods

Two hundred and ninety-three confirmed dengue patients were classified as experiencing primary, secondary or indeterminate infections using plaque reduction neutralisation tests performed 6 months after resolution of the acute illness. We developed and validated regression models to differentiate primary from secondary dengue on multiple acute illness days, using Panbio Indirect IgG and in-house capture IgG and IgM ELISA measurements performed on over 1000 serial samples obtained during acute illness.

Results

Cut-offs derived for the various parameters demonstrated progressive change (positively or negatively) by day of illness. Using these time varying cut-offs it was possible to determine whether an infection was primary or secondary on single specimens, with acceptable performance. The model using Panbio Indirect IgG responses and including an interaction with illness day showed the best performance throughout, although with some decline in performance later in infection. Models based on in-house capture IgG levels, and the IgM/IgG ratio, also performed well, though conversely performance improved later in infection.

Conclusions

For all assays, the best fitting models estimated a different cut-off value for different days of illness, confirming how rapidly the immune response changes during acute dengue. The optimal choice of assay will vary depending on circumstance. Although the Panbio Indirect IgG model performs best early on, the IgM/IgG capture ratio may be preferred later in the illness course.

Background

Dengue is the most widely distributed mosquito-borne human viral disease and represents a major public health burden globally. An estimated 390 million infections occur each year, of which around 100 million are symptomatic [1]. Although the majority of symptomatic individuals recover after a short illness, a small proportion of patients develop severe complications that can be life threatening. There are four dengue viral serotypes (DENV1–4), all of which may cause severe disease. Although the pathogenesis of severe dengue remains incompletely understood, it is clear that following an initial (primary) infection with one viral serotype, a subsequent infection with a different serotype (secondary infection) is more likely to result in severe disease [2, 3]. A number of virological and immunological parameters that are thought to contribute to dengue pathogenesis differ between individuals with primary and secondary infections [4,5,6,7], and differentiating between primary and secondary dengue is important for pathogenesis and epidemiological research. However, it also has potential utility in clinical practice, especially early in the disease evolution when knowledge of the immune status of a confirmed dengue case could help clinicians decide on the need for hospitalisation or frequency of follow-up, and might improve the performance of risk prediction algorithms for severe disease.

Several serological methods have been developed to categorise dengue infections as either primary or secondary. Although Haemagglutination Inhibition Assays (HI assays) have been traditionally considered the gold standard, the technique is complicated and time-consuming to perform, and requires experienced technical staff and samples collected in late convalescence. By comparison, serological testing using ELISA (enzyme-linked immunosorbent assay) techniques to measure IgM and/or IgG levels is technically much simpler, and a number of algorithms now use ELISA titres measured on a single specimen to define immune status. However, the range of techniques and definitions used is highly variable including: Capture IgM/IgG ratios greater than 1.78 [8], 1.2 [9] and 1.4 [10] to define primary infections; Capture IgG/IgM ratios greater than 1.10 [11] and 1.14 [12] to define secondary infections; NS1 specific IgG titres [13], absolute IgG titres [14] and/or measures of IgG avidity [15,16,17]; and Indirect IgG ELISA results [18]. In most of these reported studies only small numbers of participants were involved and the data used to define the outcome (primary versus secondary disease), and to develop the corresponding diagnostic algorithm, relied on serological tests performed on the same samples; while the specific tests were usually different, some degree of linkage is inevitable given the coordinated nature of immune responses to infection within individuals. In addition the available algorithms have only occasionally accounted for the known kinetics of antibody responses during acute infection [19, 20]. A combined approach has sometimes been adopted, in one study a case was defined as primary if the IgM/IgG ratio was above 1.78, secondary if the ratio was under 1.2 and indeterminate if it fell between these values [21].

Recently, the various laboratory methods used to diagnose dengue were evaluated to define the best approach within specified time-periods during infection [22, 23]. The aim of this study was to characterise the influence of illness day on a variety of methods currently used to determine immune status in confirmed dengue cases. If applicable, we also wished to develop simple and practical models to differentiate primary from secondary dengue on specimens obtained at any time during the acute illness. To avoid the circularity mentioned above, we elected to use plaque reduction neutralisation tests (PRNTs) performed 6 months after the acute illness episode to define immune status.

Methods

Patients and samples

Laboratory confirmed dengue patients aged 5–25 years who had enrolled into one of several clinical studies carried out at the Hospital for Tropical Diseases (HTD) in Ho Chi Minh City [24, 25] were invited to participate in this study. This work, and all the associated research studies, were approved by the Ethical Committee at HTD and the Oxford Tropical Research Ethics Committee. The original studies had included both outpatient and inpatient recruitment and focused on early enrolment of children and young adults presenting with clinical syndromes consistent with dengue. In all studies, clinical information was recorded daily and a 1 ml research blood sample was obtained each day until defervescence, or, for hospitalized individuals, until discharge. Illness day was counted from 1 (Day1, Day2, Day3 etc.) where Day1 was the reported day of fever onset.

Study participants with confirmed dengue (positive dengue RT-PCR [26]) were contacted and invited to attend a follow-up visit approximately 6 months after their acute illness. Following written consent by the patient or their parent/guardian, a short questionnaire was completed, and individuals who had been well since the initial acute dengue study were asked to provide a 2 ml blood sample.

ELISA tests

Daily plasma samples obtained during the acute illness were assayed using the Panbio anti-dengue Indirect IgG ELISA, as per the manufacturer’s instructions, as well as in-house anti-dengue IgG and IgM capture ELISAs, as described elsewhere [24, 27]. For the in-house assays, the negative control was a pool of plasma from dengue naïve individuals, and the positive control was a pool of plasma from acute dengue patients. The cut-off for positivity was defined as 5 times the negative control value. The index unit was calculated as 10 times the ratio of the normalized value and the cut-off [24].

PRNT assay and rule for differentiation of primary and secondary dengue

PRNT assays were conducted on the blood samples obtained 6 months after infection. PRNTs were performed at the Laboratory of Infectious Diseases, National Institute for Allergy and Infectious Disease (NIAID), National Institutes of Health (NIH), USA, as described elsewhere [28, 29]. PRNT₆₀ titres (the reciprocal of the dilution at which the number of plaques was reduced by 60%) were used. A case was classified as primary when the PRNT₆₀ titre to the infecting serotype (i.e. causing the recent infection) was > = 20 and the PRNT₆₀ titre for all remaining serotypes was < 20. Cases were classified as secondary when the PRNT₆₀ titre for the infecting serotype was > = 20 with the PRNT₆₀ titre for at least one other serotype either > = 40 or > = the PRNT₆₀ titre of the infecting serotype (whichever was lower). Other cases were classified as indeterminate.

Statistical analysis

Cases with a clearly defined immune status according to PRNT were included for development of the algorithms. The outcome assessed was the ability to differentiate between primary and secondary infections, as defined by the convalescent PRNT titres, using only acute serology results. Models were developed for each marker separately - Panbio Indirect IgG, in-house capture IgG, in-house capture IgM, and in-house capture IgM/IgG ratio. The “all-inclusive” logistic models used the (log2 transformed) marker value and the corresponding illness day as predictors and included data from all available samples from Day2 to Day7 of illness. To account for within-patient correlation of measurements from the same patient on different illness days, a marginal logistic regression with independence working correlation was used. For each all-inclusive model, a cut-off-point was chosen as the value with the highest accuracy ([true positive + true negative]/total cases).

Potential interactions between marker levels and illness day, non-linearity of marker effects (modelled as flexible natural cubic spline functions with 4 degrees of freedom), or the effect of other covariates (sex, age, serotype, hospitalization and time of admission) were assessed using Wald-type tests. If adding an interaction term between marker level and illness day substantially improved model performance, it was added to the respective “all-inclusive model”.

The area under the ROC curve (AUC), accuracy, specificity, sensitivity, positive predictive value and negative predictive value (based on the cut-off that maximized accuracy) were used in assessing and comparing model performance. As the model complexity of the “all-inclusive” models was low compared to the amount of available data, we expected minimal over-fitting and therefore report apparent performance throughout. In addition, the all-inclusive models (including the cut-off selection) were validated using bootstrapping as described by Harrell [30], and temporal cross-validation. For temporal cross-validation, patients were divided into 5 equal groups by date of enrolment. The models were then developed on training sets including 4 of these patient groups, and validated on the remaining (5th) group of patients. The final performance was calculated as average performance of 5 validation assessments.

The performance of the all-inclusive models was also compared to various “time-specific” univariable and multivariable logistic regression models that utilised only marker values from Day3 (early-phase models), or Day6 (late-phase models), or both Day3 and Day6 (dual-phase models), and lastly all days from Day3 to Day6 together (Day3–6 models). In addition, several of the algorithms currently in use were assessed [10, 13].

All statistical analyses were performed in R version 3.1.1 (07/10/2014) [31]. Marginal logistic regression models were fitted with the R package geepack version 1.2–0 [32].

Results

Patient characteristics and associations with immune status

Three hundred three confirmed dengue patients agreed to participate in this study. One patient was excluded because of missing PRNT titres, and 9 patients were excluded because the infecting serotype was not determined, leaving 293 patients included in the analysis. The median time to sampling for PRNT assay was 209 (IQR = 188–241) days from onset of fever. PRNT results by immune status and infecting serotype are summarised in Fig. 1; 105/293 (36%) and 144/293 (49%) patients were defined as having primary and secondary infections respectively. The remaining 44 patients (15%) could not be classified.

Fig. 1

Summary of PRNT results and the classification of immune status. For each immune status category the PRNT titres are summarized in terms of median (IQR), with the current serotype shown in bold. In the blank cells titres were either unmeasurable or low (≥ 10 < 20)

Demographic and virological information is presented in Table 1. Using univariate logistic regression, older age and female sex were more likely to be associated with secondary dengue (OR (95%CI) = 1.11(1.05–1.17) for each year of age and 2.79(1.61–4.83) for female patients, see Table 1). Male and female patients were not different in age. In addition, infections with DENV2 and DENV4 were more likely to be secondary than DENV1, with OR (95%CI) compared to DENV1 of 3.03(1.57–5.84) for DENV2 and 16.36(4.79–55.88) for DENV4. Illness day at presentation was not significantly associated with immune status (p = 0.1, logistic regression). Surprisingly, a higher proportion of secondary infections was seen among community (outpatient) participants compared to those hospitalised (67% versus 50%). However this finding likely reflects demographic differences in the different study populations: overall, patients in the community studies were older than patients in the hospital-based studies with a median (IQR) age of 15(12, 20) years compared to 12(10, 14) years.

Table 1 Patient characteristics. Continuous variables are presented as median (IQR) and categorical variables are presented as number and percentage

The kinetics of the antibody responses by illness day by immune status are shown in Fig. 2. IgG levels started to rise earlier and peaked at a higher level in secondary compared to primary infections, while IgM kinetics were quite similar.

Fig. 2

Antibody kinetics by immune status. Each thin line is an individual patient values, coloured by immune status group. The thick coloured line for each colour is the smoothed median for the relevant immune status group. The black horizontal lines indicate the cut-off for a positive result for each test

The all-inclusive models

For each of the different markers separately (results of IgG capture, IgM/IgG capture ratio and IgG indirect) the all-inclusive model used all the values for the relevant marker obtained from each of Day2 to Day7 (541 and 673 samples from 105 primary and 144 secondary infections, respectively). The all-inclusive models were developed using marginal regression, clustered by patient, with the following formula: Logit (secondary/primary) = Marker + DOI. By the Wald-type test, adding an interaction term between marker level and illness day, adjusting for other covariates, including serotype, or including the marker in a flexible, potentially non-linear way were all shown to have a significant effect in all of the all-inclusive models (all p-values< 0.001). However, the general performance of the various models changed very little, except for the model using the Panbio Indirect IgG with the interaction term. In this instance, the performance of the model with the interaction was better on Day6, accuracy of 0.77 and 0.84 for non-interaction and interaction models respectively (Table 4). Taking into account these effects, we chose three linear models for subsequent analysis. These models were models including marker and illness day for a) the in-house capture IgG and b) the in-house capture IgM/IgG ratio, and c) a model including the marker and the illness day and an interaction term between the marker and illness day for the Panbio Indirect IgG.

The “all-inclusive model” based on the Panbio Indirect IgG and including the interaction term gave the best performance compared to the other models by all metrics assessed (see Table 2). The Panbio Indirect IgG model had the best AUC of 0.90 (95%CI = 0.88–0.92) and accuracy of 0.85 compared to the models with the other markers (AUC of 0.86(0.83–0.89) and accuracy of 0.84 for in-house capture IgG, and AUC of 0.88(0.85–0.90) and accuracy of 0.84 for in-house capture IgM/IgG). As expected, the model based on in-house capture IgM values was not suitable for defining immune status with an AUC of only 0.55(0.51–0.59) and an accuracy of only 0.58.

Table 2 Performance of the all-inclusive models to discriminate primary from secondary dengue

The cut-offs for each parameter on each day, as derived from the three best all-inclusive models are presented in Table 3. For the Panbio Indirect IgG, and the in-house capture IgG, an infection would be defined as secondary if the value equals or exceeds the cut-off for that day, while for the capture IgM/IgG ratio, an infection would be defined as primary if the ratio equals or exceeds the cut-off for that day. Classifying based on the cut-offs for positivity defined by the manufacturer for the indirect assay (i.e. primary classified as < 11 Panbio units), the classification would be the same as our classification on Day2, but from Day3 onwards an increasing percentage of cases (20% on Day3 up to 79% on Day7) classified as primary using our algorithm would be classified as secondary using just the manufacturer’s positivity cut-off.

Table 3 Cut-offs for the selected parameters, derived from the all-inclusive models on each individual day of illness

We also took the cut-off points for each day from the all-inclusive models and evaluated all metrics (except AUC) separately by illness day (Table 4). Panbio Indirect IgG performance proved to be better in the early compared to the late phase, with decreasing accuracy from Day2 to Day7: 0.86 to 0.79. Conversely the capture IgG and capture IgM/IgG proved to be better in the late compared to the early phase (capture IgG accuracy: Day2 to Day7: 0.76 to 0.80 and capture IgM/IgG accuracy: Day2 to 7: 0.80 to 0.83).

Table 4 Performance of the all-inclusive models on each individual day of illness, from Day2 to Day7

Validation

For both the temporal and bootstrap validation methods, the performance of all metrics for all models was very similar to their performance during the development process. For example, there was an accuracy of 0.90 from the bootstrapping validation and 0.83 from the temporal validation, compared to 0.85 in the development process (Additional file 1: Table S1). This indicates that models were not over-optimized or over-fitted.

Comparisons with time-specific models

We compared the performance of the all-inclusive models with the corresponding time-specific models: early-phase, late-phase, dual-phase and Day3–6 models. The performances of the all-inclusive models on Day3 and Day6 (Table 4) were comparable to the performances of the corresponding early-phase and late-phase models (Table 5). The performance of the dual-phase and Day3–6 models was a little better than the all-inclusive models (Table 4).

Table 5 Performance of early-phase, late-phase, dual-phase and Day3–6 models

Comparisons with selected existing algorithms

The established algorithms of Innis and Shu using the IgM/IgG ratio showed better performance in the late phase (accuracy on Day6 of 0.82 (Innis) and 0.84 (Shu)) compared to the early phase (accuracy on Day3 of 0.76 (Innis) and 0.74 (Shu)). The combined algorithm, i.e. the algorithm using both cut-offs of 1.2 and 1.78, gave good stable performance in both early and late phases (accuracy of 0.80 and 0.82 on Day3 and Day6 respectively). However 6% of cases could not be classified with this strategy.

Discussion

Differentiating between primary and secondary dengue infections is important especially in pathogenesis research and for epidemiological surveillance, but also potentially in clinical practice. In this study, we evaluated the time-course of different sero-diagnostic responses during acute dengue in 1214 daily specimens from 249 patients, and developed various models to differentiate between primary and secondary infections, using the PRNT responses 6 months after infection as gold standard. For all assays, the best fitting models estimated a different cut-off value for different days of illness, confirming how rapidly the immune response changes during acute infection.

The all-inclusive models using the Panbio Indirect IgG, in-house capture IgG, and in-house capture IgM/IgG ratio performed well, both in general over the illness course, and when derived on any day from Day2 to Day7 (accuracy of 0.8–0.85 in differentiating between primary and secondary dengue). Although the dual-phase and Day3–6 models were a little better, the cost and practical difficulties associated with additional sampling and testing, limit the relevance of this approach. We also assessed the performance of the two most widely used of the established algorithms [10, 13] using our dataset. Innis and Shu’s algorithms showed better performance in the late phase, which may reflect the time point of sample collection in the studies used to define these algorithms. The combined strategy (which has never been formally assessed), using both Innis’ and Shu’s cut-offs of 1.78 and 1.2, gave comparable performance to the all-inclusive model based on the in-house capture IgM/IgG ratio. This is consistent with the findings for the cut-offs derived from the in-house capture IgM/IgG all-inclusive model, which ranged from 1.8 on Day2 (similar to Innis) to 1.0 on Day7. Using the combined algorithm, however, immune status cannot be defined for patients where the IgM/IgG ratio falls between 1.2 and 1.78, an issue that is circumvented with the models developed here. The 1.4 cut-off used by Kuno is the same as the value we estimate on Day4, and the 1.2 used by Shu falls between our estimates from Day5 and Day6. Therefore application of Innis’ algorithm overestimates secondary infections from Day2 onwards, while both Kuno and Shu algorithms overestimate primary infections early in illness, and overestimate secondary infections later in illness. Although here we assessed our in-house capture assays, the same principles are likely to apply to commercial capture ELISA assays, particularly when considering the IgM/IgG ratio.

It is important to note however, that these models were developed using confirmed dengue cases, and further work will be needed to assess their utility when dengue is suspected but not yet confirmed and where Zika and chikungunya may be circulating. Although rapid NS1 testing is available in some clinical settings, there is usually a delay for RT-PCR confirmation. IgM and IgG responses as measured by capture ELISA did not rise above the threshold for a positive response until after Day4 in most cases (Fig. 2), and the reliability of measurements that fall below the positive threshold in any assay is questionable [10]. By contrast indirect ELISA methodology measures much lower concentrations of dengue specific IgG, which should already be present in the early days of an infection in individuals previously exposed to DENV, but not in naïve individuals. In line with this, the all-inclusive model based on Panbio Indirect IgG without the interaction with illness day showed very good performance in the early phase, but less good performance later in illness course; however the interaction term helped to improve performance at this later time point. At present, the Panbio Indirect IgG ELISA is mainly used in a qualitative way (with a positive or negative outcome) and this assessment extends the utility of the test in classifying primary and secondary infections in confirmed dengue cases.

Solely based on measures of goodness of fit, a model based on the Panbio Indirect IgG would be appropriate for early specimens (illness day≤4), while a model based on capture IgG or IgM/IgG ratios would be more suitable for late acute specimens (illness day> 4). In practice, the choice will likely depend on the setting. In intervention trials or cohort studies, where subjects are being closely monitored, individuals are more likely to be tested early in infection, so the Panbio Indirect IgG model may be preferable. However, in tertiary hospital settings, patients tend to present later in illness course, so the capture IgG or IgM/IgG ratio model is likely to be preferable. Whichever test is used, our results show that the incorporation of day of illness into the algorithm means that determining primary or secondary status can be done using a single sample on each day of acute infection.

Several potential limitations to this study need to be considered. First, we developed a large number of models on the same dataset. However, this is the most extensive assessment of its kind to date involving 249 individuals, and we planned all statistical analyses in advance. In addition we included validation using bootstrapping and temporal cross-validation to show that the reported models did not over-fit the data or lead to over-optimistic performance claims. Second, we chose to use PRNTs 6 months after infection as the gold standard for developing the models, not HI during infection as has been used conventionally in previous studies. One reason for this choice was to try to avoid circularity – i.e. use of serological data collected at the same time-point both for developing the diagnostic algorithms and for assessing outcome. In addition, although cross-reactivity is recognized as a potential issue for all serological tests for dengue, neutralization assays are considered to have the greatest specificity to differentiate between flaviviruses. Japanese Encephalitis virus (JEV) transmission occurs in Vietnam, with increasing uptake of vaccination in recent years. However since the PRNTs were dengue specific, cross-reactivity should be reduced, and there were still many individuals who were indirect IgG negative at the time of the acute illness episode, suggesting that widespread JE vaccination has not yet had a major impact. DENV neutralizing antibody level is also not thought to be influenced by prior flavivirus vaccination (yellow fever or Japanese encephalitis virus) if performed in late convalescence [33, 34].

With the recent widespread transmission of Zika virus globally, we must also consider the possibility of Zika cross-reactivity in both the acute samples and in the PRNTs. Though we know little about Zika circulation in Vietnam at the time this study was undertaken, we cannot exclude the possibility that those classified as a secondary dengue infection had instead experienced a first Zika infection. Overall however, we identified more secondary cases compared to primary, consistent with the theory that the outcome of a true secondary dengue exposure will be more severe compared to a first. However, more research is needed to understand how exposure to other flaviviruses changes the results of dengue serology and infection outcome.

Another limitation of using the PRNT assay is that the differences in titres to each serotype (generally higher titres against DENV1 compared to other serotypes) meant that more DENV2, DENV3 and DENV4 infections were classified as indeterminate immune status and therefore excluded from the analysis. Although this might bias the performance of the all-inclusive models, when we applied these models to specimens obtained during the acute illness from this indeterminate group of 54 patients, the majority (70–74%) were classified as secondary infections, in agreement with visual inspection of marker dynamics shown in Fig. 2. In addition, when using the models to define immune status for all 303 patients included in the study, 59–63% were classified as secondary dengue, a very similar proportion to the 144/249 (59%) classified as secondary dengue based only on the 6 month PRNT data. This suggests that the indeterminate group were similar to the whole study group.

Conclusions

In summary, in this study we developed diagnostic algorithms aimed at differentiating between primary and secondary dengue infections during the acute phase of illness. By describing in fine detail acute serological responses in this large group of confirmed dengue patients we have contributed to the overall knowledge of the typical patterns seen, and confirmed the rapid evolution of responses during the first week of illness. We tested models based on a variety of assays and found that models using our in-house capture IgG, the capture IgM/IgG ratio or the Panbio Indirect IgG all performed well across sequential days of illness. We illustrate that primary vs secondary discrimination on single samples from different days of illness can be improved by taking the change in titres over time into account. The findings suggest that the all-inclusive model based on the commercial Panbio Indirect IgG kit may be suitable for defining immune status in the early phase of illness, while the capture IgG or IgM/IgG models are preferable after Day4 of illness. However, more importantly the work emphasizes the variability and uncertainty surrounding use of the current algorithms. The participants in our studies were mainly children and young adults reflecting the epidemiology of symptomatic dengue in southern Vietnam; the underlying principles should be similar in other populations with different dengue endemicity, but local evaluation and adaptation of the models/cutoffs would likely be necessary before application of these models in other settings.