In our study, we described a comprehensive surveillance system including demographic, epidemiological, microbiological and clinical data. Most cases were reported in both databases with only small percentages of cases missing in either of the data sources. The completeness of the data in the notification database is high, and a high percentage of cases are notified in time, which facilitates control measures in an early stage. For data completeness, there were some significant differences at RPHS level, with individual RPHSs with a much lower completeness scores than the others. However, there was no consistent pattern regarding the best and lowest score per indicator.
For a complete evaluation of a surveillance system, it is recommended to also include more qualitative indicators like usefulness, simplicity, flexibility and acceptability . As the main objective of our study was to describe the proportion of cases reported in both systems and the quality of the data in the national surveillance system, we focused on existing data available in the national surveillance databases and therefore only on the quantitative quality indicators.
For 87% of all unique records, linking between the two sources was possible. As no common identifier was available in both databases, the linking was done probabilistically, based on basic demographics and (approximate) date of diagnosis. Manual linking is very labor-intensive, especially in a period with high numbers of reported cases and is probably also less accurate compared with using a common unique identifier (deterministic linking). Introducing a common identifier could therefore facilitate the linking process at the RIVM.
For 73% of the OSIRIS only cases, it was clear from comments that no isolate was sent to the NRLBM for serogrouping, or serogrouping was done in another hospital, so these would still have been missing with deterministic linking. For the 128 NRLBM only cases, the exact reasons for not reporting in OSIRIS are unknown. It is possible that the patient did not comply with the clinical notification criteria, although it is hard to imagine a patient with N. meningitidis isolated from a normally sterile site without having clinical symptoms like fever. Another explanation could be a failure (technical or human) by the regional laboratories to notify a case to the RPHS. Based on the comments in OSIRIS, in 28 cases (1.4% of all OSIRIS cases) no notification was made by the regional laboratory to the RPHS, before the RIVM contacted the RPHS. A third explanation could be a failure at the RPHS level to notify a case in OSIRIS. Since recently the RIVM contacts the RPHSs actively when a case with known residence is reported by the NRLBM, without a matching case in OSIRIS, to get better insight into reasons for not reporting in OSIRIS.
The 87% of cases that is reported in both databases is high compared with some comparable studies from other European countries with similar IMD surveillance systems. In an Irish study, looking at the surveillance data from 1999 to 2015, for 83% of all unique records a link could be made between the disease reporting system and the laboratory surveillance system . From the disease reporting database, 87% of the cases were also reported in the laboratory database, from the laboratory database 94% of the cases were also reported in the disease reporting database. In a German study, looking at the year 2003 only, the percentage of linked cases was 61% . In this study, the number of cases in the disease reporting system was much higher than in the laboratory surveillance system. Based on a two-source capture-recapture analysis in this study, the sensitivity of the disease reporting system was high with 90%. In our study, we decided not to perform a capture-recapture analysis, as the two sources are not independent; the isolate positive for meningococcus at the regional laboratory, is the starting point for both the notification to the RPHS as sending in material to the NRLBM.
The data completeness of OSIRIS cases is high, with serogroup information entered in 87% of the cases. When we limit the analysis to cases that are reported in the NRLBM database, this percentage increases to 91%. When we further restrict to recent cases (2013 and later), for 98% of the cases that are reported in both databases, information on the serogroup is reported. The high percentage in serogroup completeness is also the result of policy of the RIVM, in which the RIVM actively requests to enter the serogroup information, when a notification is sent for finalization in OSIRIS. Since 2006, for 38% of all OSIRIS cases, such a request has been made. The percentage of requests per year was not significantly different between the years.
For vaccination status, the completeness of data is high, although the improvement we have seen for serogroup in recent years is not seen for this variable. We expect the validity of the information entered by the RPHSs is high, as these request the vaccination status (type of vaccine and date of vaccination) of each case from a regional office of the national vaccination registry in most cases. To monitor the effect of the new vaccination schedule, it is important to report the vaccination status, especially when the serogroup of a case is known. With recent media attention for the increasing numbers of IMD-cases caused by serogroup W (IMDW) in the Netherlands and the new vaccination schedule and therefore increasing awareness for IMD, we expect the completeness of this variable will increase in the next years. Furthermore, as the OSIRIS questionnaire has been updated because of the new vaccination policy, the question has been brought under attention of public health professionals again.
Data completeness for country of infection and mortality have both been high since the implementation of OSIRIS in 2003. The decreasing number of deaths for IMD since 2004 is also in line with the official death records (based on ICD10 code: A39), but the numbers are slightly higher in OSIRIS . In the official death records, the higher number of deaths in 2016 in ORISIS (13 deaths, of which 7 in IMDW cases) is not seen, which might be the result of official death records that are finalized before the diagnosis IMD is confirmed. The true number of deaths due to IMD might also be higher than reported in OSIRIS, as some of the cases might die after the finalization of the record in OSIRIS. With the higher mortality among IMDW cases shown in the UK (12%) and in the Netherlands (17%), the question on mortality remains important [10, 11]. In light of the more non-specific presentation of cases with IMDW [12, 13], it might be useful to focus more on clinical symptoms. Through the years, questions on clinical manifestations have varied a lot and have also been absent for many years before reintroduced using drop down answer menus with standardized and more detailed formats in 2015. The completeness for the clinical manifestation variables in the years 2015 and 2016 was 100%. Since 2017, detailed questions on disease symptoms, including gastro-intestinal symptoms, are included in the questionnaire as well. Information on sequelae is not included in the OSIRIS-database as the questions on clinical symptoms focus on the symptoms on the day of onset. The main task of the RPHSs is to implement control measures as early as possible, for which a detailed follow-up on one single patient is not needed. Therefore the follow-up period on cases is short and case records are finalized early in OSIRIS.
Timely notification is important to enable the RPHS to provide post exposure prophylaxis to close contacts and to deal with unrest among close contacts in case of severe illness or death. The timeliness of notifications to the RPHS and from RPHS to the national level was good, with 86 and 98% notified on time respectively, with a significantly higher percentage in recent years for the notification to RPHSs. One major limitation in the calculation of timeliness is the entry of dates. Only the date of notification to the national level is entered automatically, by creating the notification in OSIRIS. All other date values are entered manually by the RPHS, which is prone to typing errors. For example, for 131 cases, we calculated a negative time between diagnosis and notification. Furthermore, there is no clear definition of the day of diagnosis (probable clinical diagnosis or lab confirmation) and the day of notification to the RPHS. Some RPHSs use the day mentioned on the lab result, other RPHSs use the day the lab result has arrived or been processed. In the latter case, reporting delay from diagnosis to notification might be missed. In practice, the timeliness of control measures (chemoprophylaxis and vaccination) is more important than the timeliness of notification. The RPHSs report their control measures only in their own case management systems. As all RPHSs in the Netherlands are separate organizations, the data from these case management systems are not available to others. Therefore we have no insight in the timeliness of the control measures on regional level.
For all of the four key indicators for completeness as well as for the timeliness, some individual RPHSs had a significantly lower score than others. For the four completeness indicators, it was not the same RPHS scoring lowest. When stratifying for the three time periods also different RPHSs scored lowest per time period per indicator. The lower completeness seems therefore not a structural problem, but still we think it remains important to pay attention on data completeness. The results of our completeness analysis will therefore be shared with the individual RPHSs. For timeliness, already a system is in place, in which the RIVM sends a report to RPHSs with information on their timeliness of all notified cases in OSIRIS. This helps RPHSs to monitor trends over time and to investigate structural problems in their notification procedures .