Oops, sold out…

Buy two, get one free, says your supermarket’s promotional folder. Great news! There’s a promotion on your favourite wine and it starts tomorrow. The following day you head straight for the shop in your lunch break, only to find that the shelves are already empty. Oh well, they must have miscalculated the stock. I bet that feels familiar”, says Shari De Baets with a grin. She is a doctoral research associate who recently obtained her PhD based on research that could help prevent such situations.

“How are sales figures, and thus the stocks required, predicted for promotional activities? Usually with a simple statistical model – and a simple one at that – with an expert forecaster who adjusts the results of this model as he or she sees fit”, she explains. “The question is: how do you make the most accurate predictions? Should you use a model, or is it a good idea for experts to fine-tune things here and there?”

The Vlerick Forecasting Research Centre was founded just as Shari embarked on her doctoral research. Among other things, the centre analyses the impact of human judgment on forecasts in general, not only for exceptional situations such as promotions.

The best of both worlds

For the first part of her research, Shari studied data from a company that distributes magazines. It used a statistical model that was fairly good at predicting the normal circulation of each magazine. However, when there were promotions – a free music CD with the purchase of a specific magazine, for example – this model was less accurate and one of the expert forecasters needed to adjust the figures. However, Shari’s data analysis revealed that the model’s forecasts were also adjusted unnecessarily.

“That confirmed the findings in academic literature: people feel they have no choice but to make changes – even the slightest ones – because otherwise it seems as if the computer has taken over their job. These unnecessary changes need to be prevented. It is not enough simply to say that they shouldn’t be made, or make it technically impossible to change things: people are really creative when it comes to bending the rules. Consequently, we have developed a new statistical model that takes account of the adjustments made by the expert forecaster to differing degrees, depending on the accuracy of past predictions. The benefit of the new model is that it leaves scope for human intervention while boosting the accuracy of new forecasts.”

One is better than none

In the second part of her research, Shari conducted an experimental study to assess whether the predictions made by a statistical model influence the prognoses of expert forecasters and if so, how. The participants in the experiment were presented with a graph showing the past sales figures for a specific product, as well as the related promotional campaigns and their effect on the sales figures. Then they were asked to predict the turnover in normal circumstances and if there was a promotion. They were divided into four groups for this:

  • Group 1 worked without a statistical model.
  • Group 2 used a basic model that made forecasts based on past data without ‘knowing’ which details were linked to a promotional campaign.
  • Group 3 used a model that could distinguish between regular sales figures and promotions, but only predicted normal sales accurately. The model did not include a formula for promotional sales.
  • Group 4 was given a very advanced model that not only distinguished between promotional periods and normal sales periods, but also included special formulas to accurately predict both normal turnover and promotional sales figures.

“Since we were working in an experimental setting, we were able to determine in advance what the turnover would be and develop the formulas in such a way that the prognosis was precisely accurate”, Shari explains.

 “We were expecting group 1 to make the least accurate forecasts and group 4 the most accurate. We also expected group 3 to fare better than group 2, but the latter prediction proved not to be entirely accurate. In fact there was no difference whatsoever between the results using the basic model (group 2) and those using the model that could accurately forecast regular sales figures but not promotions (group 3). So it turns out that even the most basic of models is better than human predictions alone.”

No more need for experts?

Can we interpret these results as a plea for using statistical models instead of human judgment? “Don’t be fooled. We never said that human input should be entirely done away with, on the contrary. An expert forecaster still adds considerable value, especially in the case of promotions. For statistical models to make accurate forecasts, they require sufficient measuring points, i.e. past data. However, promotions tend to be the exception to the rule, and above all: every promotion is different. An expert who is familiar with his or her market and target audience is better at predicting the popularity of that free music CD, extra magazine or voucher. So should we go for the human or the model? I would say, use both, but in the right way,” Shari concludes.

In a nutshell

Key findings from Shari’s research for organisations looking to come up with better forecasts:

 

1. Do not be afraid to use statistical models, no matter how basic they are.
Good software packages are available that will make your life easier, and they will only increase the accuracy of your predictions.

2. Ensure your expert forecasters are well trained.
Teach them how to use statistical models and also give them   insight into the mistakes that are made when the results of models are adjusted, so they know when it is better just to rely on the model.

3. Do not underestimate the value that an expert forecaster adds.
Especially for the launch of new products or actions, when not much prior data is available, experts are in a better position to assess the situation.

Source: ‘Allowing for promotion effects in forecasting: Effects of judgment and formal forecasts.’ by Shari De Baets. Doctorate in Applied Economic Sciences, obtained at Ghent University in 2017. Supervisors: Professor Dirk Buyens (Ghent University and Vlerick Business School) and Professor Karlien Vanderheyden (Vlerick Business School).

Accreditations
& Rankings

Equis Association of MBAs AACSB Financial Times