A forecast does not improve on its own. You have to measure how far off it is, and whether it is systematically too high or too low. Three measures cover most of it.
Example
One product over three months:
| Month | Actual sales | Forecast | Deviation |
|---|---|---|---|
| August | 100 | 90 | 10 |
| September | 80 | 100 | 20 |
| October | 120 | 110 | 10 |
| Total | 300 | 300 | 40 |
MAPE
MAPE is the average percentage deviation per month:
(10/100 + 20/80 + 10/120) / 3 = (10% + 25% + 8.3%) / 3 = 14.4%
MAPE is easy to understand, but goes wrong on products with low sales. If a product sells 2 units and the forecast was 6, the deviation is 200%, even though it is only 4 units. And if the product sells 0 units, MAPE cannot be calculated at all. It therefore suits webshops with many low-selling products poorly.
WAPE
WAPE adds up the deviations and divides by total sales:
40 / 300 = 13.3%
Large products weigh more, and products with low sales cannot push the figure out of proportion. It is probably the most useful measure if you want to see the accuracy for a supplier or the whole range.
Bias
Bias shows whether the forecast is systematically above or below:
(−10 + 20 − 10) / 300 = 0%
In the example the forecast misses, but equally in both directions. If bias is +15%, you order too much every time, and it ends up as overstock. If it is −15%, products run out. Bias is often more important than the deviation itself, because a skewed forecast can be corrected, while random noise cannot.
Test on the history
To compare methods, you can run a backtest:
- Hide the last three months of the history.
- Let each method forecast those three months from the rest.
- Compare with actual sales and calculate WAPE and bias.
- Use the method that performed best, and repeat the test regularly.
Three months is a compromise. Shorter periods are too random, and longer periods say less about how the product sells now.
What is good enough
It depends on the product. A stable A product can often be forecast within 15 to 25%, while a product that sells a few units a month rarely gets below 50%. It probably makes more sense to follow the development over time and look at bias than to chase a specific figure.
Backtest in lesslater
lesslater runs a backtest on the last three months for each product. Three months is too few to switch method on a small difference, so lesslater only switches away from the seasonal profile, or from SBA on products that sell rarely, when another method performed much better. The analysis shows how far off the forecast has been.