Naked Forecasts Considered Harmful

Here at Lokad, our commitment is to deliver the best forecasts that technology can provide. As a result, potential clients often ask if we can provide forecasts alone instead of a full managed solution. In this episode of LokadTV, we explain why these “naked forecasts” can introduce a whole host of different problems and how, even with better forecasts, a practitioner can actually end up degrading the performance of a supply chain.

Many clients request naked time-series forecasts from Lokad. Conceptually, time-series forecasting is something very simple, with time-series being both the input and output of the forecast, yet it’s very seldom the best way to solve supply chain issues, despite their popularity.

When Lokad first started out in 2008 delivering these naked forecasts, they simply weren’t working effectively. The problem wasn’t statistical, the metrics were correct, with a very low error margin and high accuracy in regards to Mean Absolute Percentage Error, Absolute Error and Mean Square Error, etc. The forecasts Lokad were producing were arguably state-of-the-art, so just why didn’t they work as they should have?

It took a large European retailer back in 2011, who organised a benchmark for forecasting solution vendors, for Lokad to realise what was missing. This retailer set a problem involving forecasting demand for mini-markets, each with 5000 products, getting replenished twice a week. Lokad managed to produce results that were 20% more accurate than the other vendors based on the metrics set for the benchmark… by forecasting zero demand… thus zero replenishment and zero sales. This is what happens with more accurate forecasts expressed only in percentages of error: you get nonsense.

We go on to explain why it’s not the average that is of interest for any business but the extreme scenarios, the factors that create stock-outs and vast excesses of stock. A classical, naked forecast doesn’t look at these extremes, it looks at the middle ground, the “average”.

To wrap things up, we discuss the real-world supply chain challenges that businesses face and how probabilistic forecasts can help to properly apprehend these challenges. We also go into more detail on why Excel isn’t equipped to deal with probabilistic data.

Timestamps

00:08 Introduction

00:29 What do you mean by “naked forecasts”?

02:24 Is there an appetite for solutions of this nature?

03:39 Why didn’t it work?

07:53 Why is it something companies still want?

10:47 Is that what you moved towards them?

12:46 Why can’t you produce forecasts that look at those extreme scenarios?

14:31 Large corporations have internal S&OP teams who can work with these forecasts. What are the challenges there?

16:19 If we are giving them a whole range of possible futures, can they work with that?

19:01 From a technical perspective, how easy would it be to manipulate these pieces of data?

20:26 To sum up today’s episode, we could say that having a good forecast that takes into account all the extremes is important, but more important is what you do with this forecast. Is that right?