Table 3 List of main Error Measures. Arithmetic mean and absolute errors are used to calculate these measures in which positive and negative deviations do not cancel each other out and measures do not provide any information about the direction of errors
Measure name | Formula | Description | Scaled | Outlier Protection | Other forms | Penalize extreme deviation | Other Specification |
|---|---|---|---|---|---|---|---|
Mean Absolute Error (MAE) | \( MAE=\frac {1}{T} \sum _{t=1}^{T} |e_{t}| \) | Demonstrates the magnitude of overall error | No | Not Good | GMAE | No | - |
Root Mean Squared Error (RMSE) | \( RMSE= \sqrt {\frac {\sum _{t=1}^{T} e_{t}^{2}}{T} } \) | Root square of average squared error | No | Not Good | MSE | Yes | - |
Mean Absolute Percentage Error (MAPE) | \( MAPE=\frac {1}{T} \sum _{t=1}^{T} |\frac {e_{t}}{y_{t}}| \) | Measures the average of absolute percentage error | Yes | Not Good | MdAPE a, RMSPE b | No | - |
symmetric Mean Absolute Percentage Error (sMAPE) | \( sMAPE=\frac {2}{T} \sum _{t=1}^{T} |\frac {e_{t}}{y_{t}+x_{t}}| \) | Scale the error by dividing it by the average of y t and x t | Yes | Good | MdsAPE | No | Less possibility of division by zero rather than MAPE. |
Mean Absolute Relative Error (MARE) | \( MARE=\frac {1}{T} \sum _{t=1}^{T} |\frac {e_{t}}{e_{RWt}}| \) | Measures the average ratio of absolute error to Random walk error | Yes | Fair | MdRAE, GMRAE | No | - |
Relative Measures: e.g. RelMAE (RMAE) | \(RMAE=\frac {MAE}{MAE_{RW}}= \frac {\sum _{t=1}^{T} |e_{t}|}{\sum _{t=1}^{T} |e_{RWt|} }\) | Ratio of accumulation of errors to cumulative error of Random Walk method | Yes | Not Good | No | - | |
Mean Absolute Scaled Error (MASE) | \( MASE=\frac {1}{T} \sum _{t=1}^{T} |\frac {e_{t}}{\frac {1}{T-1}\times \sum _{i=2}^{T}|y_{i}-y_{i-1}|}| \) | Measures the average ratio of error to average error of one-step Random Walk method | Yes | Fair | RMSSE | No | - |
Percent Better (PB) | \( PB=\frac {1}{T} \sum _{t=1}^{T} [I\{e_{t},e_{RW_{t}}\}]\) | Demonstrates average number of times that method overcomes the Random Walk method | Yes | Good | - | No | Not good for calibration and close competitive methods. |
| Â | \( |e_{s,t}|\leq |e_{RW_{t}}| \leftrightarrow I\{e_{t},e_{RW_{t}}\}=1 \) | Â | Â | Â | Â | Â | Â |
Mean Arctangent Absolute Percentage Error (MAAPE) | \( MAAPE=\frac {1}{T} \sum _{t=1}^{T} arctan|\frac {e_{t}}{y_{t}}| \) | Calculates the average arctangent of absolute percentage error | Yes | Good | MdAAPE | No | Smooths large errors. Solve division by zero problem. |
Normalized Mean Squared Error (NMSE) | \( NMSE=\frac {MSE}{\sigma ^{2}} = \frac {1}{\sigma ^{2} T} \sum _{t=1}^{T} e_{t}^{2} \) | Normalized version of MSE: value of error is balanced | No | Not Good | NA | No | Balanced error by dividing by variance of real data. |