The focus of this study is on the clinical utility of two commercial NAATs in the aetiological diagnosis of patients with suspected sepsis. We found that the multiplex PCR assay added diagnostic value by timely detection of clinically relevant microbes missed by blood culture. The results of the microarray-based assay corresponded well to those of blood culture, but its clinical utility is reduced by the prerequisite of time-consuming cultivation.

Currently, blood culture is considered the gold standard for aetiological diagnosis of sepsis although it suffers from having a low sensitivity. It can only detect viable microorganisms, and the medium is not optimized for culturing fungi and fastidious bacteria. Thus, blood culture can be considered to be a poor gold standard, which implies difficulties in evaluating novel sepsis tests since no other laboratory reference standard for sepsis diagnosis exists. For that reason, different reference standards have been used in studies assessing sepsis tests. Some have used blood culture results alone as gold standard [20,28], whereas others have considered all pathogenic findings detected by any method [16,29]. We assessed the performances of blood culture and the two NAATs by classifying all findings judged as clinically relevant as “true positives”. The diagnostic sensitivity for multiplex PCR (64%, Table 2) was comparable with the rate in a similar study by Carrara et al. (65%, p = 0.91) [16], as well as the specificity (96% vs. 92%, p = 0.05) [16]. However, Loonen et al. have reported significantly lower sensitivity (37%, p < 0.001) and specificity (77%, p = 0.0001) for Magicplex™ [19]. For the microarray assay, we observed a significantly lower diagnostic sensitivity (62% vs. 96%, p < 0.0001, Table 2) whereas the specificity was equal (99% vs. 99%, p = 1.0) compared with a previous study [20]. The use of different gold standards and study populations may explain observed differences in the performance characteristics [16,19,20,30].

Although the multiplex PCR assay could not detect all clinically relevant microbes, it offered added diagnostic value by the detection of several important pathogens not detected by the conventional culture-based method (Table 3). Consequently, the concordance between the results of blood culture and multiplex PCR assay is moderate (kappa 0.50) (Table 2), whereas the microarray results showed a high degree of agreement with the results of blood culture (kappa 0.90) (Table 2). These results could be expected since the microarray assay is performed on aliquots derived from blood culture bottles, whereas the multiplex PCR is performed on microbial DNA extracted from a different whole blood sample than the cultured portion.

A remarkable finding is that multiplex PCR of only 1 mL whole blood reached a sensitivity of 64% compared to 72% for culture using 32–40 mL blood, despite more stringent criteria for clinical relevance in the PCR case. However, both methods failed to detect a number of sepsis cases. From a strictly quantitative aspect, the detection level for microorganisms in blood samples at a given time, the diagnostic sensitivity, is depending on the concentration of microorganisms and the volume sampled. The extent of variation in the yield of bacteria in time is not known. In bacteraemia, an average concentration of 0.25 CFU/mL blood was reported by Arpi et al. [31]. Jonsson et al. [32] theoretically calculated the probability of detecting bacteria as a function of the concentration in blood and found empirically by blood culture that 29% of all cases with Escherichia coli and 18% of S. aureus bacteraemia had a most probable concentration of only 0.036 CFU/mL. The gain in yield of microorganisms by increasing the volume of cultured blood in more modern automated culture systems was emphasized by Cockerhill et al. [33] and Lee et al. [34]. To obtain a >99% sensitivity with these systems, four blood cultures, each involving 20 mL is needed [34]. The gain is probably due to the detection of the most minute concentrations, either by enhanced culture systems or overcoming the effects of early antibiotic therapy by resins.

NAATs introduced to diagnose sepsis by sampling blood may add new information implying higher analytical sensitivity. By definition, the analytical sensitivity of an assay refers to the smallest value of the analyte that can be resolved with a given degree of confidence and is not synonymous with the diagnostic sensitivity. A single bacterial cell is obviously the smallest theoretical unit that can be applied for blood culture or culture in a wider sense. With NAATs, a single bacterial cell, dead or alive, may contain multiples of a certain target sequence, e.g., in bacteria the conserved 16S rRNA gene sequence is often used as PCR template due to its high copy number in each cell. The downside of using detection methods with high analytical sensitivity, such as PCR, is the increased number of findings of unknown significance as well as contaminants. We then have to cope with unexpected microbial findings that do not necessarily corroborate with the clinical picture, as translocation of bacteria and fungi over the mucosal gut barrier in patients with malignancies or on parenteral nutrition is an increasing diagnostic dilemma. Thus, we are urged to tighten the communication between the laboratory and the clinicians to organise, assemble and critically review the unexpected findings.

There is also a qualitative aspect. Certain species seem to correlate better with genuine sepsis than others, i.e., Enterobacteriaceae spp. and pneumococci [35]. The detection of identical microorganisms from multiple sampling sites or occasions is another genuine marker for true bacteraemia. This further implies that a single finding of a microbe with one detection system, e.g., culture, may move the interpretation from probable to proven if the same finding is done with e.g., PCR. However, true bacteraemia also occurs in patients void of an inflammatory response. This shall not be regarded as a benign finding until the following factors are ruled out. First, translocation of bacteria from the gut to the bloodstream may occur in patients with malfunctioning mucosal barriers. Well known examples are E. cloacae (occult malignancy) [35], Streptococcus group G, i.e., Streptococcus dysgalactiae subsp. equisimilis (haematological malignancy and solid tumours) [36], and the former Streptococcus bovis group, i.e., S. gallolyticus subsp. gallolyticus and subsp. pasteurianus (colonic cancer), and S. infantarius (cancer of the bile tree or pancreas) [37]. Experimental work on animals showed an increased risk for bacterial translocation for subjects fed exclusively by the parenteral route [38]. Finally, there are a large number of conditions linked to immunodeficiencies where both mucosal barriers and normal inflammatory response are malfunctioning. A significant microbial finding, e.g., single detection of a species with significant pathogenic profile or multiple detection of the same but low-pathogenic species, should alert the clinician independent of clinical signs of sepsis and whether the detection method was culture or molecular. Combining the methods might therefore provide important clinical information concerning not only the acute infection but also underlying conditions.

In our laboratory, it usually takes around 6–7 hours after a blood culture bottle has flagged positive before the isolate has grown enough on the plates enabling species identification within minutes by MALDI-TOF MS (Figure 2). At the same time a primary reading of the antibiogram is done and aids to disclose resistant strains of Staphylococcus aureus, Enterobacteriaceae spp. and non-fermenters. However, the time needed for species identification by conventional culture-based methods differs between clinical laboratories depending on routines. For the microarray, the turnaround time from positive bottle to microorganism identification was about four hours including preparation of blood culture bottles and DNA extraction. We estimated that the use of the microarray would save 2–3 hours compared to routine methods, but it is more labour intensive. For both blood culture and microarray, the incubation time of typically 1–3 days must also be considered. However, a time saving was obtained using multiplex PCR with an estimated turnaround time of only seven hours since this assay was performed directly on whole blood sample requiring no incubation time.

For the multiplex PCR assay, the whole process control was invalid or partially invalid in 45 (12%) of the 383 blood samples initially tested. According to the manufacturer, the whole process control indicates whether the process has functioned optimally or not. No further information is given when the whole process control is flagged as invalid. We speculated that a very high DNA concentration in the samples either inhibited the PCR reaction or the purification process. Therefore, these samples were retested on a ten-fold dilution of the extracted DNA and all but one sample then gave a final result. The high rate of invalid whole process control thus reduces the clinical utility of multiplex PCR since such samples need to be retested.

Our study has several limitations. One of the most important is that TATs were not precisely measured, just roughly estimated, mainly due to handling procedures. In addition, none of the NAATs were run 24/7; multiplex PCR was performed once daily whereas the microarray was ran every second day.