Two questions about prediction accuracy

It is often asked that demand forecasts are more accurate. The answer is that it depends on your industry. For example, in the industrial product field, with small quantities and many varieties, the accuracy of 60% prediction may be quite good; but in the automobile manufacturing, household appliances, and consumer goods industries, such accuracy may mean disaster.

How accurate is the forecast accuracy?

In the United States, a company called ToolsGroup has done research and said that the forecast accuracy of consumer goods is about 85%, retail sales are similar to consumer goods, and industrial goods are much lower, only 70%. This should mainly be for North American companies, using a monthly forecast.

According to Gartner's survey, consumer product companies have forecast accuracy of between 50% and 60%. This figure is significantly lower than the ToolsGroup statistics. However, Gartner's statistical method is rather strict. It is based on the SKU and location level. The statistical method is the percentage of absolute difference (MAPE), ie, the accuracy of the prediction = (1-(|actual-predictive |) / actual value) *100%[2].

In fact, the accuracy of the predictions should be considered high, and its significance is not as large as some people think. In our opinion, focusing on the prediction accuracy, it is more important to analyze deviations, understand the causes of deviations, and take corrective measures to improve the accuracy of future predictions.

It is also worth noting here that the accuracy of forecasting is a multi-dimensional concept and it must be clear about the details: (1) The product level or the SKU level? Product-level forecasting accuracy can be high, while the SKU level is much lower; (2) at the factory level or at the channel level? The factory level can be very high, but sales are more focused on specific channels. (3) How long is the forecast in advance as a benchmark? Is M-1 (predicted last month) or M-2, M-3?

Businesses also often make a fuss about time units. The smaller the time unit, the lower the accuracy of the prediction, and vice versa. The smaller the better the unit of production hopes, the better it is, if it is not, because it is the way they arrange resources. No longer, Zhou. The requirements for sales are not as detailed as monthly. But for production, the first and last days of the month, the first week and the last week can be quite different. Therefore, monthly statistics are still far from the daily production schedule for solving production.

For the supply chain, SKU-level forecasting accuracy is best. However, the accuracy at this level is generally very low. Some of the companies we are familiar with are only 20% or 30%. For some products, from the point of view of the production process, the difference at the SKU level is small, and the accuracy of the statistics at this level may not be of great significance if the resources spent for the evaluation are taken into account.

Prediction is not measurement accuracy, but deviation rate

The forecast is like gambling. What does the gambling show? Explain that you are lucky. We are actually very difficult to analyze the reasons for gambling, in addition to the basic "start from the data, the end of the judgment" demand forecasting process, as well as the selection of a more appropriate mathematical statistics model.

For demand forecasting, adherence to the basic principle is to select the most appropriate statistical model and follow the basic process of “starting with data and ending with judgment”. And to avoid the failure, we must find out from the predicted deviation rate—all predictions are wrong. Identifying deviations and correcting differences are the process of adjusting predictions. Demand forecasting is like launching a missile. It flies all the way and corrects it all the way. The process of commanding a target is actually the process of correcting the deviation.

Let us first see how the deviation is calculated. The concept is very simple: the difference between the actual value and the predicted value is the deviation. But forecasting is a cyclical update process. If you scroll monthly, there will be a forecast every month. Which month's forecast should be used to calculate the deviation? This depends on the response cycle of the supply chain: In the general industry, the production cycle plus the purchase lead period of long-period materials is about 3 months. Therefore, in theory, using the forecast three months ago (M-3) is the most reasonable: In January, you told me that the forecast for April is 100, and I follow this to purchase and produce. As a result, the actual demand in April is 80, so the deviation is 20, and the deviation rate is 25%.

M-3's bias rate is generally very high, revealing too many "problems." The biggest problem is that the forecast is too early and accurate. The solution is to shorten the response cycle of the supply chain so that we do not need to make predictions as early as possible. So time is the biggest factor, and it is not controllable for demand forecasting. Instead, it has masked many other controllable factors.

Some companies use M-1 to measure deviations, using the one-month forecast as a benchmark. This is reasonable, because in the general industry, the freeze period for demand forecast is probably within one month (which means that the quantity, configuration, and date cannot be changed). If it enters the frozen period, it also changes the forecast, that is, the production and supply chain. The price is very high, so it must be strictly controlled. The half-freeze period lasts about 1 to 2 months. During this period of time, the demand changes. I'm sorry, you have to think of ways to digest the supply chain. In this way, the problems in the frozen period will be paid by the sales; the changes in the semi-freezing zone will be paid by the supply chain, and the two ends will be considered as a compromise solution.

In fact, how long ago the forecast is used as a benchmark is the result of the game between sales and supply chain forces: The stronger the sales force, the more recent the forecast is selected as the benchmark; the more crude the supply chain arm, the longer it will be possible to use it. The forecast is based on benchmarks. Presumably we are now clear, if both companies say that their forecast bias is 30%, if one is using M-1 and the other is M-3, then their difference is not a little embarrassing.

Having defined how deviations are calculated, let us further explore how the differences are analyzed. In general, we are interested in several aspects: absolute deviation (absolute value), relative deviation (percentage), and direction of deviation (prejudice).

Let's talk about absolute deviation first. Assume that the demand forecast is 100 and the actual demand is 80, then the absolute deviation is 20. Divide the actual demand by 80 to get the absolute percentage deviation of 25%. Absolute percentage makes different products comparable, so it is the more common one in the analysis of differences. However, you can see that the absolute percentage hides the direction of deviation and cannot judge whether the prediction is imaginary or hypothetical. For example, the demand forecast is 60, the actual demand is 80, and the percentage difference is also 25%. But you know, this is a prediction that is low and it will create a shortage, which is not the same as the surplus caused by the forecast 100 and the actual 80. The impact on the supply chain is also different.

This is why some companies take the absolute value and directly calculate the predicted and actual percentage deviation. This deviation can clearly determine the direction of deviation between prediction and actuality to reveal systemic bias in demand forecasting. For example, in the “End of Judgment”, there are several sales people who habitually overestimate demand, while other sales personnel are high and low, which is more in line with statistical rules. Based on this analysis, targeted organizational measures can be taken to correct the behavior of these salespersons.

Some companies accumulate predictions and actual deviations to obtain cumulative deviations to determine the cumulative effect of the forecast. For example, the January, February, and March forecasts are high, and the April, May, and June months are too low. The additions in the six months may cancel out, resulting in no sluggish inventory. However, this does not mean perfect. For example, based on the overestimation of the first three months, the supply chain may have to rush to work, resulting in additional operating costs; due to the low forecast in the last three months, the factory may dismiss part of its employees and cause severance costs. It can be said that once aggregated, we may risk losing certain information and masking certain issues. Therefore, the predicted cumulative deviation is generally not used independently, but is used together with other deviation indicators.

In addition, there are many statistical methods of deviation, such as the variance and mean square error between the statistical prediction and the actual value, to measure the magnitude and dispersion of the deviation. However, no matter what kind of statistical method is used, if the value exceeds a certain limit, it will become an abnormal value, which requires root cause analysis and corrective measures. Although demand forecasting is "prediction," most of the time, it is correcting deviations: finding differences, analyzing differences, understanding differences, taking correction measures, such as replacing statistical models, meeting with sales, marketing, product management, and discussing related functions. It is through the handling of “exceptions” one by one that we are able to avoid failure and thus succeed.

(Note: Liu Baohong, author of the best-seller of Supply Chain Management, founder of "Supply Chain Management Column" and an MBA from Arizona State University, USA. His best-selling books include "Supply Chain Management: Solutions for High Cost, High Inventory, Heavy Assets," " Purchasing and supply chain management: A practitioner's point of view, "Supply Chain Management: A Road for Experts in Practitioners." He has been researching and practising supply chain management in the US for more than a decade, often traveling between China and the United States. Local procurement, planning, and supply chain management talent to help local companies improve their purchasing and supply chain management levels. To contact him, email bob. or visit his website () to find the latest training information.)