The Mean Absolute Percentage Error (MAPE) is a widely used metric for evaluating the accuracy of a forecasting model. Despite its popularity, there are scenarios where MAPE can yield extremely high values, leading to misleading interpretations of forecasting accuracy. In this article, we delve into the technical reasons why MAPE can be disproportionately high and explore examples, key points, and enhancements for its interpretation.

Understanding MAPE

MAPE is defined as the average of the absolute percentage errors between the forecasted and actual values. Mathematically, it is expressed as:

$$\text{MAPE} = \frac{100}{n} \sum\_{t=1}^{n} \left| \frac{A\_t - F\_t}{A\_t} \right|$$

where: • $A_t$ is the actual value at time $t$, • $F_t$ is the forecasted value at time $t$, • $n$ is the total number of observations.

The formula essentially computes the average percentage error by which forecasts differ from actual values. However, this metric has several pitfalls that can cause it to be excessively high.

Key Reasons for High MAPE Values

1. Zero or Near-Zero Actual Values: • When the actual values $A_t$ are zero or close to zero, the percentage error term $\frac{A_t - F_t}{A_t}$ becomes undefined or extremely large. Even small forecast errors can lead to very high MAPE values in such cases.
2. Skewed Data Distributions: • If the distribution of actual values is heavily skewed with a small number of very small values, these small values can disproportionately inflate the MAPE. The presence of outliers can also exacerbate this effect.
3. Magnitude of Forecast Errors: • Larger forecast errors naturally lead to high MAPE values. However, large errors relative to the scale of the actual values can cause this distortion.
4. Sensitivity to Data Scale: • MAPE is scale-sensitive. It disproportionately emphasizes periods or observations with smaller actual values than larger ones, leading to a skewed perception of forecast accuracy.
5. Non-symmetric Nature: • MAPE is not symmetric. This means positive errors (over-forecasts) and negative errors (under-forecasts) do not offset each other, often resulting in a high MAPE, especially for datasets with frequent swings or volatility.

Example Illustration

Consider the following scenario:

In this table, although the absolute errors are consistent across observations, the MAPE is distorted greatly by the zero actual value, showing an undefined or infinitely large error.

Enhancing the Interpretation of MAPE

To counteract the issues with MAPE, several strategies can be employed:

Alternative Metrics

• Symmetric MAPE (sMAPE): • This variant of MAPE uses the average of the absolute differences divided by the sum of actuals and forecasts, reducing the impact of small denominators.

$$\text{sMAPE} = \frac{100}{n} \sum\_{t=1}^{n} \frac{|A\_t - F\_t|}{(|A\_t| + |F\_t|)/2}$$

• Mean Absolute Error (MAE): • Unlike MAPE, MAE depicts errors in actual units, not percentages, thus avoiding misinterpretation due to data scaling issues.

$$\text{MAE} = \frac{1}{n} \sum\_{t=1}^{n} |A\_t - F\_t|$$

Data Transformation

• Log Transformation: • Applying a log transformation to the data can stabilize variance and allow MAPE to be used with less risk of distortion due to small values.

Weighting Adjustments

• Error Weighting: • Prioritize large actual values when calculating MAPE by applying a weighting scheme to reduce the impact of very small actual values.

Conclusion

While MAPE is a straightforward and popular forecasting accuracy metric, it is crucial to understand its limitations and interpret its results with caution, particularly in datasets with zero or near-zero actual values, skewed distributions, or large errors. Alternating between MAPE and other metrics like sMAPE or MAE can provide a more comprehensive evaluation of forecast accuracy. Consideration of the context and nature of your data will ensure a more reliable interpretation of the forecasting model's performance.