Fig. 5

Median, 25th and 75th, and interquartile ranges for the difference between WIS scores when fitting a performance based ensemble (PB) and equally weighted ensemble (EW) paired by survey for three different ensembles: an ensemble that includes only computational models (blue), only human judgment (red), and a chimeric ensemble that includes both computational and human judgement models (gold). A “spotty memory” strategy was used along with five imputation techniques for training. Ensemble predictions are stratified by A  incident cases and B deaths. For the majority of imputation techniques used for predictions of incident cases, training a performance based ensemble shows similar results for a chimeric, computational, and human judgement ensemble. For deaths, performance based training improves predictions of a computational ensemble, shows little improvement to a chimeric ensemble, and weakens predictions of a human judgment ensemble