Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Performance of prediction rules and guidelines in detecting serious bacterial infections among Tanzanian febrile children

BMC Infectious Diseases volume 19, Article number: 769 (2019) Cite this article

Abstract

Background

Health-workers in developing countries rely on clinical algorithms, such as the Integrated Management of Childhood Illnesses (IMCI), for the management of patients, including diagnosis of serious bacterial infections (SBI). The diagnostic accuracy of IMCI in detecting children with SBI is unknown. Prediction rules and guidelines for SBI from well-resourced countries at outpatient level may help to improve current guidelines; however, their diagnostic performance has not been evaluated in resource-limited countries, where clinical conditions, access to care, and diagnostic capacity differ. The aim of this study was to estimate the diagnostic accuracy of existing prediction rules and clinical guidelines in identifying children with SBI in a cohort of febrile children attending outpatient health facilities in Tanzania.

Methods

Structured literature review to identify available prediction rules and guidelines aimed at detecting SBI and retrospective, external validation on a dataset containing 1005 febrile Tanzanian children with acute infections. The reference standard, SBI, was established based on rigorous clinical and microbiological criteria.

Results

Four prediction rules and five guidelines, including IMCI, could be validated. All examined rules and guidelines had insufficient diagnostic accuracy for ruling-in or ruling-out SBI with positive and negative likelihood ratios ranging from 1.04–1.87 to 0.47–0.92, respectively. IMCI had a sensitivity of 36.7% (95% CI 29.4–44.6%) at a specificity of 70.3% (67.1–73.4%). Rules that use a combination of clinical and laboratory testing had better performance compared to rules and guidelines using only clinical and or laboratory elements.

Conclusions

Currently applied guidelines for managing children with febrile illness have insufficient diagnostic accuracy in detecting children with SBI. Revised clinical algorithms including simple point-of-care tests with improved accuracy for detecting SBI targeting in tropical resource-poor settings are needed. They should undergo careful external validation against clinical outcome before implementation, given the inherent limitations of gold standards for SBI.

Peer Review reports

Background

Acute febrile illnesses are the most common presentation of young children attending outpatient settings worldwide [1]. Like in well-resourced settings, the majority of acute febrile illnesses are caused by viral pathogens requiring minimal supportive intervention; serious bacterial infections (SBI) have become increasingly rare with improving vaccination coverage and hygiene [2, 3]. However, the lack of adequate diagnostic tools makes it difficult to differentiate these viral diseases from the minority of children with SBI. Children with serious bacterial infections (SBI) often present with non-specific clinical signs and several concomitant symptoms [4]. Sub-standard management of children with infections has resulted in persistent high mortality from common childhood infections [5] and high-volume over-prescription of antibiotics [6].

Health workers rely on the World Health Organization (WHO) Integrated Management of Childhood Illnesses (IMCI) algorithm, which recommends presumptive treatment based on clinical signs and symptoms (besides the rapid diagnostic test for malaria that was introduced in the 2014 version, [7]). The Integrated Community Case Management guidelines (iCCM) is a simplified version of IMCI, geared towards community health workers [8, 9]. Due to the lack of available evidence when IMCI was initially developed, the algorithm was based mainly on expert opinion in addition to small derivation studies [10]. Though IMCI and iCCM have been implemented globally, their performance in detecting children with SBI has not been validated to date using stringent microbiological methods, instead of expert clinical diagnosis (and chest radiograph (CXR) in some studies) [11, 12]. Adherence to IMCI has been low. The reasons for nonadherence to IMCI are numerous and complex [13, 14], but one important aspect is the content: for example, IMCI lacks guidance in key areas, e.g. for patients with fever without clinical focus [7]. As a result, clinicians over-prescribe antibiotics out of the fear of missing patients with SBI [15]. Therefore, there is a need to improve current management guidelines for the primary care management of acute febrile illnesses, including evidence from economically developed countries. Here, a series of clinical and laboratory prediction rules and clinical guidelines, with different degrees of validation, have been designed for the management of febrile children in the ambulatory setting [16,17,18,19,20,21,22,23,24,25,26,27]. There is a growing body of evidence that the causes of acute non-malaria febrile illnesses in children in low- and high resource settings are in fact quite similar [2]: cosmopolitan viruses and bacteria are the causative agent in the vast majority of cases while tropical pathogens cause only a minority of febrile episodes at the outpatient level. Clinical signs and laboratory tests from such clinical prediction rules and guidelines developed in well-resourced countries may thus also be useful for detecting SBI in children in low-resource settings. However, external validation to support their use in resource-poor settings is lacking. This is especially important because of differences in clinical presentations (e.g. malaria co-infection), the health care system (e.g. access to care, the possibility of safety netting, the level of training of primary care providers).

Methods

Aim

The aim of this study was to estimate the diagnostic accuracy of existing prediction rules and clinical guidelines, including IMCI and iCCM, in identifying children with SBI in a resource-poor setting.

Design

We performed an external, retrospective validation study of existing prediction rules and guidelines on a dataset collected prospectively in Tanzania that contains children aged 2 months to 10 years with fever presenting to outpatient care [2].

Participants/ setting

The study population comprised 1005 children from a study on causes of fever in rural and urban Tanzania, the ‘Tanzanian Fever Study’ [2]. Briefly, children aged 2 months to 10 years with fever (axillary temperature of ≥38 °C) were enrolled consecutively at two outpatient clinics in 2008. Children with severe acute malnutrition and/or those requiring immediate live-saving procedures were excluded. This was partly for safety reasons, but also because WHO recommends antibiotic treatment for all febrile children with severe acute malnutrition as these patients have a distinct immune response putting them at high risk of SBI [7, 28]. All participants in the dataset, including children with malaria infection, were included into the validation exercise. We performed sensitivity analyses to assess the influence of malaria co-infection on the diagnostic performance (see below).

Outcome definition

The outcome, SBI, i.e. a bacterial infection requiring antibiotic treatment, was defined as presence of one of the following: bacteremia (positive blood culture for a known pathogen), Salmonella typhi infection (positive blood-or stool culture, or positive specific IgM rapid diagnostic test), radiographic pneumonia, urinary tract infection (positive urine dipstick and urine culture), meningitis, bacterial gastroenteritis (positive stool culture), significant skin/soft tissue infections and other systemic bacterial infections not routinely detected by blood culture (rickettsiosis, coxiellosis, and leptospirosis). Definitions were based on the methodology used in the ‘Tanzania Fever Study’: for each patient, the final diagnosis (or diagnoses) was established with a computer-generated algorithm based on pre-defined clinical and microbiological criteria [2]. These criteria were derived from international guidelines as well as systematic reviews.

Clinical and laboratory assessment

Investigators used standardized case report forms to record clinical findings, including 23 symptoms and their respective duration, potential travel history and/or sick contacts, known chronic conditions, and 49 clinical signs. At the initial visit a systematic set of investigations was performed according to predefined algorithm; malaria testing was done for all children [2]. If a clinical or laboratory diagnosis could not be made at the initial visit, a follow-up visit was scheduled for day 7 that included a full clinical and laboratory assessment for patients with persistent symptoms. In all cases, blood samples and pooled nasal and throat swabs were taken for microbiologic testing (cultures and rapid tests) on site and further serologic and molecular work-up in Switzerland and the USA. A complete blood cell count, including white blood cell count was done on site for all children. C-reactive protein (CRP) and procalcitonin (PCT) were performed retrospectively on stored samples by ELISA as detailed elsewhere [29]. CXR were performed in the subgroup of cases fulfilling the WHO clinical definition of pneumonia [30]. The diagnosis of radiological pneumonia was made in cases where CXR showed ‘primary endpoint consolidation’ according to WHO’s Pneumococcal Trialist Ad Hoc Committee recommendations [31]. If the IMCI clinical criteria for a suspected human immunodeficiency virus (HIV) infection were present, voluntary HIV testing was recommended to the child’s guardian.

Selection of prediction rules and guidelines

All available prediction rules (laboratory and clinical) for identifying any SBI in children in the outpatient settings were identified through a structured literature review in Medline and Embase as part of the development of a novel disease management algorithm [32]. The search strategy is detailed in the Additional file 1 of the publication. The search was modified based on previously published systematic review and a European validation study [16, 33]. Prediction rules and guidelines that target the neonatal period, i.e. < 3 months, were excluded. We also did not include prediction rules that primarily aim at predicting death (such as the PEDIA [34], LODS [35], and SICK [36] scores) or the need for referral to the pediatric intensive care unit at in-patient level. Scores aimed at identifying dehydration for patients with gastroenteritis, or at detecting children with meningitis (there were only 2 patients with meningitis) were also not included. When variables of the dataset were not entirely matching the variables of the original rule or guideline, we identified proxies where possible (Additional file 2: Table S1). When more than 20% of the required variables were not recorded in the dataset (systematically missing), the rule/guideline was not included in the validation. This was based on the assumption that missing systematically more than 20% of predictor variables was not clinically sensible. Missing data on variables used in the validation were not imputed because the necessary missing-at-random assumption was likely to be incorrect given that all data was collected based on a predefined algorithm. We report the number of observations available for analysis of each prediction rule after application of the above assumptions. Where rules generated sum scores, previously published cut-offs were applied.

Statistical analysis

We used the Standard for Reporting of Diagnostic Accuracy (STARD) guidelines for study reporting [37]. The accuracy of the included prediction rules and guidelines was estimated retrospectively in the prospectively collected ‘Tanzania Fever Study’ dataset by calculating sensitivity, specificity, and likelihood ratio (LR). For the low prevalence outpatient setting we considered a score helpful to rule-in SBI if, when positive, they substantially raised the probability of SBI (LR+ greater 5). Scores were deemed helpful for ruling-out SBI if, when negative, they substantially lowered the probability of illness (LR- lower than 0.2).

Clinical features were deemed warning signs if, when positive, they substantially raised the probability of illness—i.e., positive likelihood ratio of more than 5.0. Clinical features were deemed rule-out signs if, when negative, they substantially lowered the probability of illness—i.e., negative likelihood ratio of less than 0.2.

We performed the following sensitivity analyses by comparing the 95% confidence intervals (CIs) of diagnostic accuracy measures: First, to assess the influence of age range, we compared the target age group of the rules/ guidelines with those of the entire validation dataset. Second, as some predictors (fast breathing in IMCI, iCCM, and ALMANACH, and a positive CXR in the American Academy of Emergency Physicians [AAEP] guideline) were part of the diagnostic criteria for pneumonia in the validation dataset, we compared the full dataset with a dataset excluding pneumonia cases for these 4 guidelines. The same was done for UTI for prediction scales and guidelines that use urinary dipstick (Bleeker Score, Lab Score, ALMANACH and AAEP). Third, since malaria is known to raise CRP values [38], we compared malaria negative patients with the full dataset for prediction rules that contain CRP. Fourth, for prediction rules that were originally derived for children with fever without source, we compared the full dataset with the dataset containing children with fever without source only. All analyses were performed with Stata version 13.1. The confidence intervals were calculated using the Stata diagt procedure (http://www.stata.com/stb/stb59/sbe36_1/diagt.hlp. We used a web-based tool to generate Venn diagrams (http://jura.wi.mit.edu/bioc/tools/venn.php).

Results

Prediction rules and guidelines

Through the structured literature review [32], we identified 34 prediction rules/guidelines for the use in febrile children. Sixteen were designed to predict SBI at the outpatient level (Fig. 1, Tables 1 and 2).

Fig. 1
figure 1

Flowchart of scores identified and considered for validation (adapted from [32]). Pediatric intensive care unit (PICU)

Full size image
Table 1 Clinical and laboratory prediction rules for management of acute febrile illnesses in childrena
Full size table
Table 2 Guidelines for management of acute febrile illnesses in childrena
Full size table

The NICE guideline is intended to predict ‘serious disease’ among children with acute febrile illness, and not to indicate antibiotic treatment. However, given that it was the only guideline designed for the use by healthcare professionals in primary care with various levels of training, we decided to include it in the validation exercise. In addition to the prediction rules and guidelines from the systematic review and European validation study [16, 33], we found one additional prediction rule for diagnosis of SBI [21], two prediction rules for pneumonia [24, 25], and four clinical guidelines (AAEP, IMCI, iCCM, and ALMANACH [7, 8, 27, 41]). ALMANACH is an improved IMCI-based algorithm that includes urinary dipstick testing [9]. Additional file 2: Table S1 displays whether the prediction rules and guidelines could be used for retrospective validation, as well as proxies for certain predictor variables used. For the prediction rules, validation was possible for the Bleeker Score, Thayyil Score, Lab Score and the Rotterdam Fever Model. More than 20% of predictors were missing systematically for other prediction rules, including 3 pneumonia rules. All clinical guidelines identified could be used for validation. Table 3 displays the prediction rules and guidelines that could be included into validation exercise. It also details the categories of SBI that were considered for the initial derivation or development of each rule/guideline.

Table 3 Prediction rules and guidelines that could be used for validation and SBI considered for each rule in the original derivation study/ at development
Full size table

Validation dataset

The full details on the demographic and clinical characteristics of the study population are provided in the original study report [2]. A SBI was identified in 16% (162/1005) of patients in the validation dataset (Table 4).

Table 4 Cross table of serious bacterial infection (SBI) categories
Full size table

Validation results

The diagnostic accuracy for all included prediction rules and guidelines was low to moderate (Table 5). The Bleeker rule, Rotterdam Fever Model (2.5% risk cutoff), and NICE guidelines had the highest sensitivity, ranging from 77.3 to 83.7%. However, the specificity of the Bleeker score was only 40.8% (95% CI 36.9–44.9%), and those of the Rotterdam Fever Model (2.5% risk cutoff), and NICE guidelines even lower: 35.6% (95% CI 32.4–39.0%) and 25.2% (95% CI 22.6–28.6%), respectively. IMCI (like iCCM) had a very low sensitivity of 37.0% (95% CI 29.4–44.6%) and a moderate specificity of 70.3% (95% CI 67.1–73.4%). Compared to IMCI, ALMANACH had a higher sensitivity of 63.3% (55.4–70.6%). However, ALMANACH’s specificity was lower compared to IMCI (63.2, 95% CI 59.8–66.4%). None of the scores had LRs that would be considered helpful for ruling-in or ruling-out SBI in low-prevalence settings (LR+ greater 5 or LR- lower than 0.2).

Table 5 results of external validation of prediction rules and guidelines to rule-in and rule-out serious bacterial infection
Full size table

Figure 2 illustrates the overlap between SBI classification (reference) and antibiotic treatment classifications by the score. The Bleeker score and NICE guideline achieved the highest proportion of correct classifications (14% of the total population) but at the expense of many unnecessary antibiotic prescriptions: 49 and 62% of patients, respectively. IMCI, iCCM and the Thayyil score resulted in the lowest proportion of correct classifications (6% of patients).

Fig. 2
figure 2

Overlap of serious bacterial infection classification (blue) and antibiotic treatment classification per rule or guideline (pink). The blue circles represent the percentage of patients with a SBI identified in the validation dataset. The pink circles illustrate the percentage of patients that tested ‘positive’ in the dataset per the rule or guideline. The overlap represents the percentage of patients with SBI who were correctly classified as such according to the rule

Full size image

Figure 3 shows the missed cases of SBI according to different classifications. Not surprisingly, IMCI, iCCM, and AAEP missed very few pneumonia cases since the classifications used by these guidelines were part of the outcome definition (see Sensitivity analyses). Similarly, missed UTI cases were fewer in scores that use urine laboratory testing. All rules and guidelines, besides the Rotterdam model at low cutoff and the NICE guideline, missed a large amount of patients with bacteremia (50–75% of bacteremia cases).

Fig. 3
figure 3

Missed cases of serious bacterial infections (SBI)

Full size image

Sensitivity analyses

Applying the rule only to the age group for which it was originally designed, resulted in a significantly higher specificity for the Bleeker rule, Thayyil score, Lab Score and AAEP guideline (Table 6). We found similar results for relevant scores when including patients without pneumonia or without malaria only, when compared to the full validation dataset (Table 6). The specificity of ALMANACH was increased when applying to patients without UTI only. There was no significant change in the performance of prediction rules originally derived for children with fever without source when we compared the full dataset with the dataset containing children with fever without source only (Table 6).

Table 6 Results of sensitivity analyses
Full size table

Discussion

In the outpatient setting in Tanzania, none of the prediction rules and guidelines examined had sufficient diagnostic accuracy to detect children with SBI. IMCI and iCCM, which were designed to be sensitive for detecting SBI in these settings, actually had very low sensitivities when applied to our validation dataset. The Bleeker score, NICE guidelines, and Rotterdam Model at low cutoff showed the highest, though moderate, sensitivity, indicating a value in ruling-out children for SBI in low-prevalence, peripheral health care settings. However, at the same time, they classified many children as having a SBI, i.e. requiring antibiotic treatment. The use of such rules or guidelines would hence require further confirmatory testing to avoid antibiotic over-prescription. Rules that use a combination of clinical and laboratory testing, the Bleeker score, Rotterdam Model, ALMANACH, and AAEP guideline had better performance compared to rules and guidelines using only clinical and or laboratory elements. We performed several sensitivity analyses to estimate whether differences in demographic and ecological characteristics between the derivation and validation population had an influence on the diagnostic accuracy. Importantly, we did not find significant differences in the performance of the SBI scores in patients of the targeted age group or patients without malaria only when compared with the entire study population.

To our knowledge, this was the first comprehensive attempt to examine the accuracy of IMCI and other prediction rules and guidelines in diagnosing SBI in a tropical, low-resource outpatient setting against a robust gold standard. Besides one 1995 study in Bangladesh that performed blood cultures and CXR [12], guidelines developed for low-resource-settings (IMCI, iCCM, ALMANACH) have never been validated against carefully established gold standards (contrary to expert opinion). Overall guidance for SBI other than pneumonia and dysentery are lacking in the current IMCI guidelines, which specifies only “to give antibiotic treatment if a bacterial source of infection is identified”. But identifying such bacterial infections without guidance is challenging for low-level health workers. Alarmingly, the sensitivity of IMCI was very low—IMCI was originally designed to be very sensitive at the expense of being specific for detection of infections requiring antibiotic treatment. The diagnostic accuracy of ALMANACH sought to address these challenges through adding urinary dipstick testing and a clinical predictor for typhoid [41]. Indeed, sensitivity was improved but at the cost of a lower specificity in our dataset. Generally, very few studies have validated outpatient prediction rules and clinical guidelines for SBI systematically. One recent study validated systematically four clinical prediction rules and two national guidelines retrospectively across datasets from primary care and emergency departments in Europe [33]. The diagnostic accuracy of the prediction rules and guidelines also validated in our study were generally higher. This may be due to the fact that the original derivation population was more similar to the validation datasets of the European validation study. Other studies in the African setting have evaluated scores for SBI and death at the inpatient level. Nadjm et al. evaluated prospectively the accuracy of WHO hospital-level clinical criteria for presumptive antibiotic treatment in detecting SBI (positive blood and/or cerebrospinal fluid culture) among 3639 admitted children in Tanzania [42]. The sensitivity was higher when compared to IMCI in our study (67.4, 95% CI 65.9–69.0%), at a lower specificity of 51.5% (95% CI 49.9–53.1%). Reported sensitivities of a similar study by Berkley at al. were even higher [43]. However, the comparison of results from these studies with the present analysis is extremely limited by the difference in prevalence of SBI in the inpatient versus outpatient setting, and the restricted number of investigations for SBI performed (blood and cerebrospinal fluid culture only). Conroy et al. validated three scores to predict in-hospital (and not outpatient) mortality among Ugandan children with fever [44]. Through mortality is a relevant and robust outcome, its use at the outpatient level, where death is a rare event, is difficult.

This study has several limitations. Only a single dataset from was available for validation, which limits the generalizability of our findings. However, rates of bacteremia in our study were similar to other studies conducted at primary care level around the same time and the dataset is likely representative of the typical case-mix [45]. There are multiple sources of heterogeneity. The most obvious one is the difference in setting for all prediction rules and two out of the four guidelines. Difference in bacterial pathogens, such as typhoid and rickettsial diseases, substantially limits the applicability of “Northern” guidelines to tropical settings. Differences in recorded values between the derivation and validation datasets is another limitation for this analysis. Though this study used robust, predefined reference criteria with extensive microbiological testing, the gold standards for SBI certainly remain imperfect [46]. For pneumonia end-point consolidation on CXR has been used though it is known that only an (unknown) percentage of consolidations are of bacterial origin, and that viral pneumonia may produce abnormalities on CXR as well [47]. As a result, test diagnostic accuracy may be biased in both directions. The diagnostic accuracy of all available tests for typhoid is poor [48] and hence the typhoid classification (combination of rapid test and blood and stool cultures), was certainly suboptimal. Consequently, the sensitivity of guidelines to detect SBI may have been underestimated. Despite the comprehensive set of clinical and laboratory predictors in the validation dataset, we were able to validate only four of the nine prediction rules plus all guidelines and had to use proxies for several predictors. For the Bleeker score, for example, “ill-appearance” was likely underestimated in our validation dataset since the variables “lethargy, and very sick child” refer to a sicker child. On the other hand, using the urine leucocyte dipstick test instead of the urine WBC likely overestimated the presence of UTI. We did not impute missing data as the “missing at random assumption” could not be assumed for the validation dataset; this may have influenced our estimates of performance for those rules that use urinary dipstick testing where we encountered a large percentage of missing data in the validation set.

Our findings have several implications for clinical practice and research in low-resource settings. First, the efforts should be made to increase the sensitivity of current screening tools for SBI. As it was intended for IMCI, clinical guidelines should have high sensitivity as the access to care in such settings is difficult, referral to higher level of care may be delayed, and safety-netting is not always available. Guidelines should be presented as stepwise decision algorithms, which follow the logical flow of the actual diagnostic process [46]. This is especially true for low-resource settings where health care providers with limited training benefit from clinical decision algorithms [49]. Within such algorithms, simple but sensitive clinical criteria will be needed to quickly rule-out children with SBI. This could then be followed by a more specific second-step laboratory testing, such as point-of-care biomarkers, in order to avoid unnecessary antibiotic treatment. However, no algorithm will have perfect diagnostic accuracy making safety netting (follow-up) an important component of clinical care. Third, disease management algorithms should undergo careful external validation before implementation. Ideally, such validation studies should be performed against clinical outcome, and not against a microbiological reference standard only as it is difficult to establish a valid microbiological reference standard. This could either be achieved through composite reference standards including clinical patient follow-up [46], or through the evaluation of decision rules through randomized clinical trials [32].

Viral infections, such as bronchiolitis, may cause severe disease. The guidance on supportive measures for viral infections by a clinical algorithm designed for the low-resource outpatient setting may be become equally important with declining prevalence of SBI. ALMANACH, for example, achieved better clinical outcome in a validation study against routine care in Tanzania [50].

Conclusions

None of the examined prediction rules and guidelines had sufficient diagnostic accuracy to detect children with SBI in a tropical, low-resource setting. IMCI and iCCM, which were designed to be sensitive for detecting SBI in these settings, actually had very low sensitivities when applied to our validation dataset. Some prediction rules and guidelines had higher sensitivity and hence showed promise to rule-out SBI in our dataset. However, they also classified a larger number of patients as having a SBI, calling for additional second-stage testing, such as point-of care inflammatory markers, and tests for severity such as oximetry and hemoglobin. New clinical algorithms should undergo careful external validation studies against clinical outcome before implementation in routine care.

Availability of data and materials

The dataset used for this analysis is available online as detailed in the original manuscript [3].

Abbreviations

AAEP:

American Academy of Emergency Physicians

CI:

Confidence interval

CRP:

C-reactive protein

CXR:

Chest radiograph

HIV:

Human immunodeficiency virus

iCCM:

Integrated Community Case Management guidelines

IMCI:

Integrated Management of Childhood Illnesses

LR:

Likelihood ratio

SBI:

Serious bacterial infection

UTI:

Urinary tract infection

WHO:

World Health Organization

References

  1. Burton DC, Flannery B, Onyango B, Larson C, Alaii J, Zhang X, Hamel MJ, Breiman RF, Feikin DR. Healthcare-seeking behaviour for common infectious disease-related illnesses in rural Kenya: a community-based house-to-house survey. J Health Popul Nutr. 2011;29:61–70.

    Article  PubMed  PubMed Central  Google Scholar 

  2. D'Acremont V, Kilowoko M, Kyungu E, Philipina S, Sangu W, Kahama-Maro J, Lengeler C, Cherpillod P, Kaiser L, Genton B. Beyond malaria — causes of fever in outpatient Tanzanian children. N Engl J Med. 2014;370:809–17.

    Article  CAS  PubMed  Google Scholar 

  3. Elfving K, Shakely D, Andersson M, Baltzell K, Ali AS, Bachelard M, Falk KI, Ljung A, Msellem MI, Omar RS, et al. Acute uncomplicated febrile illness in children aged 2-59 months in Zanzibar – Aetiologies, antibiotic treatment and outcome. PLoS One. 2016;11:e0146054.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Horwood C, Butler LM, Vermaak K, Rollins N, Haskins L, Nkosi P, Neilands TB, Qazi S. Disease profile of children under 5 years attending primary health care clinics in a high HIV prevalence setting in South Africa. Tropical Med Int Health. 2011;16:42–52.

    Article  Google Scholar 

  5. Liu L, Oza S, Hogan D, Perin J, Rudan I, Lawn JE, Cousens S, Mathers C, Black RE. Global, regional, and national causes of child mortality in 2000–13, with projections to inform post-2015 priorities: an updated systematic analysis. Lancet. 2014;385:430–40.

    Article  PubMed  Google Scholar 

  6. Risk R, Naismith H, Burnett A, Moore SE, Cham M, Unger S. Rational prescribing in paediatrics in a resource-limited setting. Arch Dis Child. 2013;98:503–9.

    Article  PubMed  Google Scholar 

  7. World Health Organization. IMCI chart booklet. Geneva: World Health Organization; 2014.

    Google Scholar 

  8. UNICEF, World Health Organization. Integrated community case management of childhood malaria, pneumonia and diarrhoea. Kampala: World Health Organization; 2010.

  9. Rambaud-Althaus C. ALMANACH: a new electronic algorithm to promote evidence-based medicine and rational use of drugs in primary care for Tanzanian children aged 2 to 59 months; 2015.

    Google Scholar 

  10. Gove S, Tamburlini G, Molyneux E, Whitesell P, Campbell H. Development and technical basis of simplified guidelines for emergency triage assessment and treatment in developing countries. WHO Integrated Management of Childhood Illness (IMCI) referral care project. Arch Dis Child. 1999;81(6):473–7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. USAID. Synopsis: validation of outpatient IMCI guidelines. Synopsis. 1998;2:1.

    Google Scholar 

  12. Factor SH, Schillinger JA, Kalter HD, Saha S, Begum H, Hossain A, Hossain M, Dewitt V, Hanif M, Khan N, et al. Diagnosis and management of febrile children using the WHO/UNICEF guidelines for IMCI in Dhaka, Bangladesh. Bull World Health Organ. 2001;79:1096–105.

    CAS  PubMed  PubMed Central  Google Scholar 

  13. Lange S, Mwisongo A, Maestad O. Why don’t clinicians adhere more consistently to guidelines for the Integrated Management of Childhood Illness (IMCI)? Soc Sci Med. 2014;104:56–63.

    Article  PubMed  Google Scholar 

  14. Walter ND, Lyimo T, Skarbinski J, Metta E, Kahigwa E, Flannery B, Dowell SF, Abdulla S, Kachur SP. Why first-level health workers fail to follow guidelines for managing severe disease in children in the coast region, the United Republic of Tanzania. Bull World Health Organ. 2009;87:99–107.

    Article  PubMed  Google Scholar 

  15. Rambaud-Althaus C, Shao A, Samaka J, Swai N, Perri S, Kahama-Maro J, Mitchell M, D'Acremont V, Genton B. Performance of health workers using an electronic algorithm for the management of childhood illness in Tanzania: a pilot implementation study. Am J Trop Med Hyg. 2017;96(1):249–57.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Thompson M, Van den Bruel A, Verbakel J, Lakhanpaul M, Haj-Hassan T, Stevens R, Moll H, Buntinx F, Berger M, Aertgeerts B, et al. Systematic review and validation of prediction rules for identifying children with serious infections in emergency departments and urgent-access primary care. Health Technol Assess. 2012;16:1–100.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Van den Bruel A, Aertgeerts B, Bruyninckx R, Aerts M, Buntinx F. Signs and symptoms for diagnosis of serious infections in children: a prospective study in primary care. Br J Gen Pract. 2007;57:538–46.

    PubMed  PubMed Central  Google Scholar 

  18. Bleeker SE, Derksen-Lubsen G, Grobbee DE, Donders aRT, Moons KGM, Ha M. Validating and updating a prediction rule for serious bacterial infection in patients with fever without source. Acta Paediatr. 2007;96:100–4.

    Article  CAS  PubMed  Google Scholar 

  19. Thayyil S, Shenoy M, Hamaluba M, Gupta A, Frater J, Verber I. Is procalcitonin useful in early diagnosis of serious bacterial infections in children? Acta Paediatr. 2005;94:155–8.

    Article  PubMed  Google Scholar 

  20. Lacour AG, Zamora SA, Gervaix A. A score identifying serious bacterial infections in children with fever without source. Pediatr Infect Dis J. 2008;27:654–6.

    Article  PubMed  Google Scholar 

  21. Craig JC, Williams GJ, Jones M, Codarini M, Macaskill P, Hayen A, Irwig L, Fitzgerald DA, Isaacs D, Mccaskill M. The accuracy of clinical symptoms and signs for the diagnosis of serious bacterial infection in young febrile children: prospective cohort study of 15 781 febrile illnesses. BMJ. 2010;340:c1594.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Brent AJ, Lakhanpaul M, Thompson M, Collier J, Ray S, Ninis N, Levin M, MacFaul R. Risk score to stratify children with suspected serious bacterial infection: observational cohort study. Arch Dis Child. 2011;96:361–7.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Nijman RG, Vergouwe Y, Thompson M, van Veen M, van Meurs AHJ, van der Lei J, Steyerberg EW, Moll HA, Oostenbrink R, Veen MV, et al. Clinical prediction model to aid emergency doctors managing febrile children at risk of serious bacterial infections: diagnostic study. BMJ. 2013;346:–f1706.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Neuman MI, Monuteaux MC, Scully KJ, Bachur RG. Prediction of pneumonia in a pediatric emergency department. Pediatrics. 2011;128:246–53.

    Article  PubMed  Google Scholar 

  25. Bilkis MD, Gorgal N, Carbone M, Vazquez M, Albanese P, Branda MC, Alterman E, Rodriguez D, Orellana L, Pedrosa OB. Validation and development of a clinical prediction rule in clinically suspected community-acquired pneumonia. Pediatr Emerg Care. 2010;26:399–405.

    Article  PubMed  Google Scholar 

  26. National Institute of Health and Care Excellence. Feverish illness in children assessment and initial management in children yonger than 5 years. NICE Clin. Guidel. London: National Institute of Health and Care Excellence; 2013.

    Google Scholar 

  27. American College of Emergency Physicians Clinical Policies Committee. Clinical policy for children younger than three years presenting to the emergency department with fever. Ann Emerg Med. 2003;42:530–45.

    Article  Google Scholar 

  28. Page A-L, de Rekeneire N, Sayadi S, Aberrane S, Janssens A-C, Rieux C, Djibo A, Manuguerra J-C, Ducou-le-Pointe H, Grais RF, et al. Infections in children admitted with complicated severe acute malnutrition in Niger. PLoS One. 2013;8:e68699.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Erdman LK, D'Acremont V, Hayford K, Rajwans N, Kilowoko M, Kyungu E, Hongoa P, Alamo L, Streiner DL, Genton B, et al. Biomarkers of host response predict primary end-point radiological pneumonia in Tanzanian children with clinical pneumonia: a prospective cohort study. PLoS One. 2015;10:e0137592.

    Article  PubMed  PubMed Central  Google Scholar 

  30. World Health Organization. Technical bases for the WHO recommendations on the management of pneumonia in children at first-level health facilities. Geneva: World Health Organization; 1991.

  31. Cherian T, Mulholland EK, Carlin JB, Ostensen H, Amin R, de Campo M, Greenberg D, Lagos R, Lucero M, Madhi SA, et al. Standardized interpretation of paediatric chest radiographs for the diagnosis of pneumonia in epidemiological studies. Bull World Health Organ. 2005;83:353–9.

    PubMed  PubMed Central  Google Scholar 

  32. Keitel K, Kagoro F, Samaka J, Masimba J, Said Z, Temba H, Mlaganile T, Sangu W, Rambaud-Althaus C, Gervaix A, et al. A novel electronic algorithm using host biomarker point-of-care tests for the management of febrile illnesses in Tanzanian children (e-POCT): a randomized, controlled non-inferiority trial. PLoS Med. 2017;14:e1002411.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Verbakel JY, Van den Bruel A, Thompson M, Stevens R, Aertgeerts B, Oostenbrink R, Moll HA, Berger MY, Lakhanpaul M, Mant D, et al. How well do clinical prediction rules perform in identifying serious infections in acutely ill children across an international network of ambulatory care datasets? BMC Med. 2013;11:10.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Berkley JA. Prognostic indicators of early and late death in children admitted to district hospital in Kenya: cohort study. BMJ. 2003;326:361.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Helbok R, Kendjo E, Issifou S, Lackner P, Newton CR, Kombila M, Agbenyega T, Bojang K, Dietz K, Schmutzhard E, et al. The Lambaréné organ dysfunction score (LODS) is a simple clinical predictor of fatal malaria in African children. J Infect Dis. 2009;200:1834–41.

    Article  PubMed  Google Scholar 

  36. Bhal S, Tygai V, Kumar N, Sreenivas V, Puliyel JM, Jm P, Puliyel JM. Signs of inflammation in children that can kill (SICK score): preliminary prospective validation of a new non-invasive measure of severity-of-illness. J Postgrad Med. 2006;52:102–5.

    Article  CAS  PubMed  Google Scholar 

  37. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, Moher D, Rennie D, de Vet HCW, Lijmer JG. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med. 2003;138:W1–12.

    Article  PubMed  Google Scholar 

  38. Naik P, Voller A. Serum C-reactive protein levels and falciparum malaria. Trans R Soc Trop Med Hyg. 1984;78:812–3.

    Article  CAS  PubMed  Google Scholar 

  39. Mccarthy PL, Sharpe MR, Spiesel SZ, Dolan TF, Forsyth BW, DeWitt TG, Fink HD, Baron MA, Cicchetti DV, Brian W, et al. Observation scales to identify serious illness in febrile children. Pediatrics. 1982;70:802–9.

    CAS  PubMed  Google Scholar 

  40. World Health Organization. Caring for newborns and children in the community. A training course for community health workers. Geneva: World Health Organization; 2014.

  41. Rambaud-Althaus C, Shao AF, Kahama-Maro J, Genton B, D’Acremont V. Managing the sick child in the era of declining malaria transmission: development of ALMANACH, an electronic algorithm for appropriate use of antimicrobials. PLoS One. 2015;10:e0127674.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Nadjm B, Amos B, Mtove G, Ostermann J, Chonya S, Wangai H, Kimera J, Msuya W, Mtei F, Dekker D, et al. WHO guidelines for antimicrobial treatment in children admitted to hospital in an area of intense plasmodium falciparum transmission: prospective study. Br Med J. 2010;340:c1350.

    Article  Google Scholar 

  43. Berkley J, Maitland K, Mwangi I, Ngetsa C, Mwarumba S, Lowe BS, Newton CRJC, Marsh K, Scott JAG, English M. Use of clinical syndromes to target antibiotic prescribing in seriously ill children in malaria endemic area: observational study. BMJ. 2005;330:995.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Conroy AL, Hawkes M, Hayford K, Namasopo S, Opoka RO, John CC, Liles W, Kain KC. Prospective validation of pediatric disease severity scores to predict mortality in Ugandan children presenting with malaria and non-malaria febrile illness. Crit Care. 2015;19:47.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Hildenwall H, Muro F, Jansson J, Mtove G, Reyburn H, Amos B. Point-of-care assessment of C-reactive protein and white blood cell count to identify bacterial aetiologies in malaria-negative paediatric fevers in Tanzania. Tropical Med Int Health. 2017;22(3):286–93.

    Article  CAS  Google Scholar 

  46. Oostenbrink R, Thompson M, Steyerberg EW, members E. Barriers to translating diagnostic research in febrile children to clinical practice: a systematic review. Arch Dis Child. 2012;97:667–72.

    Article  PubMed  Google Scholar 

  47. Lynch T, Bialy L, Kellner JD, Osmond MH, Klassen TP, Durec T, Leicht R, Johnson DW. A systematic review on the diagnosis of pediatric bacterial pneumonia: when gold is bronze. PLoS One. 2010;5(8):e11989.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Crump JA, Sjölund-Karlsson M, Gordon MA, Parry CM. Epidemiology, clinical presentation, laboratory diagnosis, antimicrobial resistance, and antimicrobial management of invasive Salmonella infections. Clin Microbiol Rev. 2015;28:901–37.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Keitel K, D'Acremont V. Electronic clinical decision algorithms for the integrated primary care management of febrile children in low-resource settings: review of existing tools. Clin Microbiol Infect. 2018;24(8):845-55. https://doi.org/10.1016/j.cmi.2018.04.014. Epub 2018 Apr 21.

    Article  PubMed  Google Scholar 

  50. Shao AF, Rambaud-Althaus C, Samaka J, Faustine AF, Perri-Moore S, Swai N, Kahama-Maro J, Mitchell M, Genton B, D’Acremont V. New algorithm for managing childhood illness using mobile technology (ALMANACH): a controlled non-inferiority study on clinical outcome and antibiotic use in Tanzania. PLoS One. 2015;10:e0132316.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

The Fever study was supported by grants from the Swiss National Science Foundation (3270B0–109696, IZ70Z0–124023 and 32003B_127160), the Commission for International Medicine of the University Hospital of Geneva, and the Bill and Melinda Gates Foundation. No specific funding was sought for this study.

Author information

Authors and Affiliations

  1. Swiss Tropical and Public Health Institute, University of Basel, Basel, Switzerland

    Kristina Keitel, Blaise Genton & Valérie D’Acremont

  2. Department of Pediatric Emergency Medicine, University Hospital of Bern, Bern, Switzerland

    Kristina Keitel

  3. Ilala Municipality, Dar es Salaam, Tanzania

    Mary Kilowoko

  4. Tanzanian Training Centre for International Health, Ifakara, Tanzania

    Esther Kyungu

  5. Center for Primary Care and Public Health (Unisanté), University of Lausanne, Lausanne, Switzerland

    Blaise Genton & Valérie D’Acremont

  6. Infectious Diseases Service, University Hospital Lausanne, Lausanne, Switzerland

    Blaise Genton

Authors
  1. Kristina Keitel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mary Kilowoko
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Esther Kyungu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Blaise Genton
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Valérie D’Acremont
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

KK conceptualized and designed the study, carried out the analysis, and drafted the initial manuscript, and approved the final manuscript as submitted. VDA led the Fever study, created the validation dataset, participated to the design of the study and revised the manuscript. MK and EK collected the clinical data for the Fever Study and revised the manuscript. BG participated in the design and analyses of the Fever Study, and revised the manuscript. All authors approved the final manuscript as submitted and agreed to be accountable for all aspects of the work.

Corresponding author

Correspondence to Kristina Keitel.

Ethics declarations

Ethics approval and consent to participate

The protocol and related documents of the ‘Tanzanian Fever Study’ were approved by the regional ethics committee in Basel (Ethikkomission beider Basel), Switzerland, and by the National Health Research Ethics Review Committee (NHRERC), Tanzania, as detailed in the original manuscript [3]. Written informed consent was obtained from the parents/legal guardian of all children who participated in the study.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Literature search terms. (DOCX 126 kb)

Additional file 2:

Table S1. Prediction rules and guidelines used for validation and proxy variables (if applicable). (PDF 58 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Keitel, K., Kilowoko, M., Kyungu, E. et al. Performance of prediction rules and guidelines in detecting serious bacterial infections among Tanzanian febrile children. BMC Infect Dis 19, 769 (2019). https://doi.org/10.1186/s12879-019-4371-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12879-019-4371-y

Keywords

BMC Infectious Diseases

ISSN: 1471-2334

Contact us