Skip to main content

Table 3 List of main Error Measures. Arithmetic mean and absolute errors are used to calculate these measures in which positive and negative deviations do not cancel each other out and measures do not provide any information about the direction of errors

From: A framework for evaluating epidemic forecasts

Measure name Formula Description Scaled Outlier Protection Other forms Penalize extreme deviation Other Specification
Mean Absolute Error (MAE) \( MAE=\frac {1}{T} \sum _{t=1}^{T} |e_{t}| \) Demonstrates the magnitude of overall error No Not Good GMAE No -
Root Mean Squared Error (RMSE) \( RMSE= \sqrt {\frac {\sum _{t=1}^{T} e_{t}^{2}}{T} } \) Root square of average squared error No Not Good MSE Yes -
Mean Absolute Percentage Error (MAPE) \( MAPE=\frac {1}{T} \sum _{t=1}^{T} |\frac {e_{t}}{y_{t}}| \) Measures the average of absolute percentage error Yes Not Good MdAPE a, RMSPE b No -
symmetric Mean Absolute Percentage Error (sMAPE) \( sMAPE=\frac {2}{T} \sum _{t=1}^{T} |\frac {e_{t}}{y_{t}+x_{t}}| \) Scale the error by dividing it by the average of y t and x t Yes Good MdsAPE No Less possibility of division by zero rather than MAPE.
Mean Absolute Relative Error (MARE) \( MARE=\frac {1}{T} \sum _{t=1}^{T} |\frac {e_{t}}{e_{RWt}}| \) Measures the average ratio of absolute error to Random walk error Yes Fair MdRAE, GMRAE No -
Relative Measures: e.g. RelMAE (RMAE) \(RMAE=\frac {MAE}{MAE_{RW}}= \frac {\sum _{t=1}^{T} |e_{t}|}{\sum _{t=1}^{T} |e_{RWt|} }\) Ratio of accumulation of errors to cumulative error of Random Walk method Yes Not Good RelRMSE, LMR [43], RGRMSE [44] No -
Mean Absolute Scaled Error (MASE) \( MASE=\frac {1}{T} \sum _{t=1}^{T} |\frac {e_{t}}{\frac {1}{T-1}\times \sum _{i=2}^{T}|y_{i}-y_{i-1}|}| \) Measures the average ratio of error to average error of one-step Random Walk method Yes Fair RMSSE No -
Percent Better (PB) \( PB=\frac {1}{T} \sum _{t=1}^{T} [I\{e_{t},e_{RW_{t}}\}]\) Demonstrates average number of times that method overcomes the Random Walk method Yes Good - No Not good for calibration and close competitive methods.
  \( |e_{s,t}|\leq |e_{RW_{t}}| \leftrightarrow I\{e_{t},e_{RW_{t}}\}=1 \)       
Mean Arctangent Absolute Percentage Error (MAAPE) \( MAAPE=\frac {1}{T} \sum _{t=1}^{T} arctan|\frac {e_{t}}{y_{t}}| \) Calculates the average arctangent of absolute percentage error Yes Good MdAAPE No Smooths large errors. Solve division by zero problem.
Normalized Mean Squared Error (NMSE) \( NMSE=\frac {MSE}{\sigma ^{2}} = \frac {1}{\sigma ^{2} T} \sum _{t=1}^{T} e_{t}^{2} \) Normalized version of MSE: value of error is balanced No Not Good NA No Balanced error by dividing by variance of real data.
  1. aMd represent Median bRMS represent Root Mean Square