We developed and tested ICARES as an automated, real-time tool for the detection of clusters of infectious diseases. In a small pilot region, ICARES detected differences in incidence in the three groups of diseases in real time (24-h window) during the first 2 years of the project. Alert 3 and alert 6 demonstrate the ability of ICARES to detect and to monitor clusters of infectious diseases in real time.
Important strengths of ICARES are the robust diagnosis data with the minimal data set, the real-time collection and easily interpretable presentation of disease data, the historic comparison specific for each health care provider, the absence of administrative burden for medical professionals and the flexibility of the system.
Disease data should be very specific and we therefore opted in our project for definition by a medical doctor. In the Dutch health care system, doctors enter a diagnostic code in their medical record routinely. This diagnostic code most likely has a higher reliability than data used by other detection tools as Google Flu Trends and Triple S, using non-specific health indicators and proxy measures to define a syndrome [22]. In our case study, the exceedingly long lasting flu season of 2014/2015 was notified and no significant alert was generated for the mild 2013/2014 flu season. On top of that, ICARES will represent the health care consumption in possible outbreaks since all patients in ICARES did visit a medical doctor.
Another strength of ICARES is the minimal data set. Details relating to geographic mapping or age cohort are important for source detection in the early phases of a possible outbreak. The minimal data set is non-patient specific and fully respects data privacy laws. But, if required, individual hospital-patient data can be traced by the treating physician since an encrypted patient identification number can be decrypted by the principal investigator in the hospital. At GP level, the treating GP can share information by finding the cases in a possible cluster via a query in their own GP information system. Diagnostics to evaluate the cluster (and the individual patient’s illness) can be advised to treating physicians by public health care professionals. This was done during the second alert.
Daily, new data from health care providers are compared with their own historic numbers. Without significant changes in coding custom or patient population, this entails that the percentage of double coded patients or travelers would be the same in both historic group and current patients making false positive clusters for these reasons less likely.
Data acquisition and presentation on a dashboard are done daily. This contains the real-time character of ICARES enabling public health authorities to analyse clusters at an earlier stage. Other comparable systems, such as the Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE), show the difficulty in detecting an outbreak soon enough to start up control measures [23]. So far, the limited amount of small clusters detected with ICARES is insufficient to evaluate its real-time character and to determine its ability to slow the spread of infection.
As shown in the third alert with a cluster of Enterovirus encephalitis, updates on the evolution of the cluster are made available on a daily basis enabling public health care authorities to inform policy makers and public adequately.
On the other hand, when numbers of infectious diseases are not above alarm threshold, a quick scan of the dashboard is usually enough to reassure public health care authorities.
The codes used for ICARES make it possible to capture clusters of a wide range of diseases via the three selected syndromes. Even new emerging infectious diseases presenting as one of these syndromes can be detected via ICARES. To implement ICARES fully, other syndromes will be added in the future. Also, in case of newly arising possible disease associations, any other disease entity might be selected for this type of surveillance.
An important reason is that ICARES algorithm is not based on a static threshold before triggering an alert. Seasonal variations in the incidence of syndromes warrant adjusting the baseline values of syndromes. The ICARES algorithm with adjusting baseline values for seasonal variations in the incidence of syndromes, gives rise to a moving threshold for cluster detection. The pragmatic and mature SPC-based (Statistical Process Control) algorithm used in ICARES can readily be used in most generalized case studies. Various challenges arising from shortcomings of other methods have been explored by various authors [24–28]. CUSUM charts seem to adapt better to this type of analysis as they help improve the consideration of seasonal patterns as mentioned by Fricker et al. [29].
This case study has several limitations as well.
Signal-to-noise ratio was questionable during this case study with two real clusters versus six false positive alerts. Positive predictive value is therefore 0.25. Although we are not aware of any missed clusters, we cannot calculate sensitivity.
Imperfections in coding for a new patient with a non-specific syndrome may constitute reasons for low signal-to-noise ratio. This may result in false positive alerts. This is illustrated by the alert 1, 5 and 7. Other reasons for false positive alerts might be provoked by other factors contributing to a syndrome resembling an infectious disease. A sudden increase in respiratory symptoms can be attributed to a contagious viral infection but also, e.g., to a high pollen count.
The relatively small number of health care facilities and, with that, the limited regional coverage during this first 2 years of ICARES may give rise to false positive and false negative alerts.
The historic data from our GPs only cover a 1-year period and are therefore not robust. Eight-year historic hospital data might be too long as changes in care and population might make the oldest data irrelevant for upcoming cluster definition. Further work is therefore required to determine the appropriate length of history.
Currently, GP data is aggregated according to the underlying patient population data. This is not possible when considering hospitals and Out-of-Hours GP services as the exact catchment area is not known. As regional coverage broadens, assessment of this catchment area will also improve and incidence rates can be calculated for all health care facilities based on the total population in the (public health) district. As more health care facilities join the ICARES project, improved mathematical modelling to define alarm thresholds will be necessary.
Alerts are visible for public health care authorities within 24 h after the treating physician routinely enters the trigger code. General Practitioners enter the ICPC code during the first consultation, DBC/DOT codes in hospital should be entered at first patient presentation. However, DBC/DOT codes can be changed when initial diagnosis changes and whether medical doctors abide by instant coding, is unknown. This could hamper real-time detection of clusters.
ICARES is a new and unique surveillance tool in the Netherlands to detect clusters of diseases in real time. Current local detection of small clusters depends on notification by medical doctors or laboratories as is defined in the Dutch Public Health Law (Wet Publieke Gezondheid), based on the International Health Regulations (IHR) [11]. Nationwide, weekly updates of virological results are published [21] and weekly updates about patients visiting their GP with influenza-like illness are reported [30]. Automated tools for real-time detection of clusters are lacking. Systems for detection of acute hepatitis or meningoencephalitis are lacking as well.
Therefore, ICARES can improve outbreak detection in the Netherlands when used as a complement rather than a substitute for human involvement in interpreting cluster detection.
Diagnostic protocols in possible clusters have not been tested sufficiently during this project. It would be interesting to explore more disease syndromes, like food-borne diseases. This might improve its use for public health care authorities.
Further implementation of ICARES will enable cost benefit analysis. At this stage, maintenance costs are less than €10.000,- per year; daily efforts of local units of infectious disease control are minimal in case no thresholds are being exceeded. Besides time expenditure of existing staff, the development and primary piloting costs did not surpass €100,000.-.
Benefits will depend on the appearance of any clusters of infectious disease and the contribution of ICARES as a complement of surveillance tools in order to curb the outbreak.
To cite an outbreak that would have benefitted from an automated surveillance system, the current Zika epidemic in South America is an example. We could survey the illness as well as complications like microcephaly and Guillain Barre syndrome by adding diagnostic codes to ICARES.
As the project evolved, more institutions have expressed their willingness to participate. At the time of writing of this paper (22 November 2016) four hospitals, four Out-of-Hours General Practitioner services and 25 GP practices (87,380 patients) submit their consultation data daily. For GP patients, this leads to a coverage of approximately 12% in the Leiden region. There is still some way to go to improve regional coverage and robustness of data.
