00:00:07 Introduction and definition of demand sensing.
00:01:02 Joannes critiques demand sensing as a vague concept.
00:02:45 Questions about the novelty of demand sensing techniques.
00:05:01 Critique of real-time data’s impact on supply chain forecasting.
00:07:49 Argument that more data and non-traditional methods are needed.
00:08:37 Use case differences between real-time and non-real-time applications.
00:09:47 Marketing gimmicks in the software industry and their impact.
00:11:25 Differentiating between good buzzwords and shallow buzzwords.
00:14:24 Examples of other gimmicks in the supply chain industry.
00:16:02 Difficulty of memorizing buzzwords and their lack of substance.
00:16:33 Law of preservation of hype and Google Trends verification.
00:17:47 Identifying valuable buzzwords with fundamental insights.
00:18:26 Quantile forecast example and the importance of understanding fundamentals.
00:19:21 Cloud computing and its simple core concept.
Summary
In the interview, Kieran Chandler and Joannes Vermorel discuss demand sensing, a forecasting method that combines real-time data and advanced mathematical techniques. Vermorel expresses skepticism about the effectiveness of demand sensing due to its vague conceptual foundation and the lack of novel, peer-reviewed research. He advises supply chain practitioners to be wary of marketing gimmicks and focus on concepts that address fundamental issues with surprisingly obvious insights. Vermorel emphasizes the importance of understanding the substance behind buzzwords and seeking innovations that provide a solid understanding and contribute to meaningful improvements in supply chain optimization.
Extended Summary
In this interview, Kieran Chandler discusses demand sensing with Joannes Vermorel, the founder of Lokad, a software company specializing in supply chain optimization. Demand sensing is a forecasting method that combines advanced mathematical techniques with real-time information. The conversation focuses on the effectiveness of demand sensing and how to distinguish between valuable concepts and marketing gimmicks.
Vermorel starts by admitting that it is difficult to evaluate demand sensing due to the vagueness of the documents he has reviewed on the subject. He believes that demand sensing is not vaporware, as the software exists, but rather mootware—something that does not deliver on its promises and ultimately does not matter. He attributes this to the concept being more of a marketing gimmick than a substantive idea.
The key advertised attributes of demand sensing include using real-time data, incorporating data beyond traditional core data (such as past inventory movement and sales demand), and leveraging machine learning to achieve greater accuracy. Vermorel acknowledges that these individual concepts seem valid, but he questions the novelty and depth of the overall approach.
To evaluate demand sensing, Vermorel looks at the main claim made by its proponents: that it can produce more accurate forecasts. He points out that statistical forecasting is a well-established field of research, with numerous online competitions and a vibrant community constantly challenging and advancing the field. Many organizations, including prestigious research institutions and large software companies, publish papers on machine learning and statistical learning, including those focused on forecasting future demand.
However, Vermorel has not seen any significant, novel research published on demand sensing in reputable journals or conferences, which raises doubts about the concept’s legitimacy. Despite the abundance of papers and materials available on the internet regarding demand sensing, the lack of concrete, peer-reviewed research and contributions from the broader machine learning and forecasting community casts doubt on the true value of demand sensing.
Vermorel expresses skepticism about the effectiveness of demand sensing as a forecasting method, due to its vague conceptual foundation and the lack of novel, peer-reviewed research on the topic. He suggests that it may be more of a marketing gimmick than a truly innovative approach to supply chain optimization.
The conversation revolves around the concept of real-time data, its relevance to supply chain forecasting, and the marketing gimmicks surrounding it.
Vermorel explains that real-time data, which refers to data with latency below human perception (around 100 milliseconds), has become popular due to its appealing nature. However, he is skeptical about whether having real-time data offers any significant improvement in forecasting accuracy. He cites a typical software demonstration where a product with three years of history is used for six months ahead forecasting. Vermorel argues that having data from 100 milliseconds ago, as opposed to 24 hours ago, will not make much difference in this context.
Another claim often made by proponents of real-time data is the use of superior machine learning techniques. Vermorel remains skeptical, comparing it to a car manufacturer claiming to have better cars because of better physics. He does agree, though, that incorporating more data than traditional time series forecasting can lead to substantial improvements.
Vermorel asserts that real-time information in supply chain scenarios is generally overkill, as the level of granularity it provides is unnecessary for most use cases. However, he notes that real-time data could be useful for controlling fast robots in automated warehouses. He challenges the idea that real-time data can make a significant difference in supply chain planning, where forecasting typically ranges from three weeks to a year ahead.
According to Vermorel, the promotion of real-time data in supply chain forecasting is often a marketing gimmick used by software vendors who lack new ideas, expertise, or technologies. He suggests that this approach reflects a certain indifference to client problems and an arrogant assumption that clients can be easily deceived. To identify valuable concepts from mere buzzwords, he advises looking at whether various people from different fields discuss the concept. If only vendors are promoting it, it is more likely to be a marketing gimmick.
As a positive example, Vermorel cites probabilistic forecasting, which is discussed by numerous communities, including those studying climate. State-of-the-art climate research is driven by probabilistic models, indicating that this approach is based on genuine substance, unlike the marketing-driven concept of real-time data in supply chain forecasting.
Vermorel begins by explaining the concept of appeal to authority, which he believes is a strong indicator of a buzzword. He uses the example of demand sensing, where the proponents put forward the names and titles of the experts involved, rather than detailing the algorithm or math behind it.
When asked about other examples of gimmicks in the supply chain industry, Vermorel notes that there have been many throughout history. He cites IBM’s push for autonomous computing ten years ago, which turned out to be unsubstantial, and the popularity of data mining twenty years ago, which has since faded. Vermorel believes that these buzzwords often fall by the wayside due to a lack of substance, and he refers to his “law of preservation of hype,” which he explains as a constant total mass of hype that only changes when a new buzzword enters the market.
Vermorel suggests that supply chain practitioners should look out for buzzwords that seem to address a fundamental issue with a surprisingly obvious insight. He provides the example of quantile forecasts, which focus on the risks concentrated at the extreme ends rather than the middle. Vermorel believes that such simple and fundamental ideas, once understood, are hard to unlearn and can lead to significant innovations.
He continues by mentioning that practitioners should be wary of buzzwords that do not provide a clear, concise understanding of their underlying concepts. For instance, cloud computing may seem complicated, but its core concept is the availability of hardware resources on-demand. Vermorel advises that if a buzzword cannot provide a solid understanding in a few minutes, it likely lacks substance and should be disregarded.
Vermorel emphasizes the importance of recognizing and understanding the substance behind buzzwords in the supply chain industry. He encourages practitioners to focus on fundamental insights that can provide a solid understanding and contribute to meaningful innovations in their businesses.
Full Transcript
Kieran Chandler: Today we’re going to discuss how well this buzzword actually performs and also whether you can tell the difference between what is a good buzzword and actually what is a lot of marketing nonsense. So, Joannes, demand sensing, what’s your initial overview of a concept such as that?
Joannes Vermorel: It’s a difficult question actually, because I’ve reviewed many documents about demand sensing, and the best thing about those documents that I can say is that they are exceedingly vague. So, it’s very hard to attribute any specific qualities or defects to a concept that is super vague. But fundamentally, I would say that demand sensing for me is a clear illustration of maybe what we could name as not vaporware, because the software exists that is supposed to be capable of delivering demand sensing to optimize your supply chain. So, it’s not vaporware, but I would say it’s mootware, like stuff that doesn’t really matter because it won’t deliver what it promises.
Kieran Chandler: You said the ideas behind it are a little bit vague, so what’s the basic concept?
Joannes Vermorel: The concepts that are advertised, and again I am saying advertised because it’s something more akin to a marketing gimmick than anything with any depth, include using real-time data, data beyond your traditional core data, like past inventory movements or past sales demand, and using machine learning, which is quite undefined in this context, to do some novel things to get more accuracy.
Kieran Chandler: So one of the things that indicates it’s a bit of a marketing gimmick is that all of those concepts seem to stand up on their own. So what are the problems here?
Joannes Vermorel: The question is how much novelty is there in terms of depth. Let’s start by looking at the main claim of the vendors who are pushing for demand sensing technologies. The main claim is that they can get more accurate forecasts. That’s a very reasonable claim; you say I have a novel technique that can produce more accurate statistical predictions. Well, it turns out that supply chain doesn’t exist in a vacuum. There is an entire field of research on statistical forecasting, with tons of online competitions where people try models and compete. The broad field of machine learning and statistical learning is very vibrant, with challenges and public results published in conferences, journals, and papers from various sources, including private laboratories like big software companies. What I’m saying is that although there are tons of advancements in machine learning for specific tasks like forecasting future demand, I haven’t seen anything that would qualify as good novel journals or high-level conferences being published through those keywords, like demand sensing.
Kieran Chandler: Let’s talk about the content itself. There’s plenty of stuff out there on the internet if you’re doing some research regarding demand sensing, with many papers and things that seem to be kind of legitimate. So, how do you make sense of that? What’s the problem with, if we review a bit, what is being presented? So, what is being presented is first this idea of having real-time data. Why not? Yet, real-time data, first let’s qualify a bit what we mean by real-time data because it sounds cool. My perspective is real-time data is basically when the latency gets below human perception. So as a rule of thumb, if we want to clarify what we are talking about, it’s 100 milliseconds. That’s when it starts to become very close to what people would qualify as real-time.
Joannes Vermorel: Okay, let’s discuss in a supply chain or supply chain problem, the fact that you have data that is accessible, or that you have an entire pipeline of data that can process your data in less than 100 milliseconds to be real-time. Do we have any chance of having, thanks to that, something that could be of the order of magnitude of a 50% improvement in forecasting accuracy? And here, I’m very skeptical. Especially when people start to do demonstrations about the software doing demand sensing and doing forecasts. In the demo, they look at a product with three years’ worth of history, and they are doing a six-months-ahead forecast.
So, if you’re forecasting demand six months ahead, the fact that you have data that is fresh from the last 100 milliseconds or fresh from the last 24 hours, so you have data that is just lagging behind by one day, frankly speaking, it’s not going to make much difference. I’m not even sure that having data that is two days old instead of one day old is actually going to make even a one percent difference six months ahead. So, that’s where I say the real-time part of the argument feels exceedingly shallow to me.
Then, let’s look at the other part of the arguments: they claim to have superior machine learning techniques. But superior machine learning techniques – as machine learning is a super broad field – it would be like a car manufacturer saying, “Oh, we have the better car because we have better physics.” Having better physics helps, but that’s a bold claim. You need a breakthrough at the level of the physical laws. So, again, I’m skeptical when you’re making this sort of claim.
So, basically, what they have is, if I fall back to the other set of arguments, they say you need to have a forecast that uses more data than traditional methods. I would say yes, absolutely. I mean, we have discussed in this show many times the fact that naive time series forecasting gives you tons of garbage. The simplest explanation is that if you only look at your historical sales, you don’t take into account stockouts. For example, if you didn’t observe any sales because you had a stockout, you don’t want to forecast zero because you’ve observed zero. Your historical sales are not historical demand. So yes, you need to include more data on top of that, absolutely, and that can lead to very substantial improvements.
Kieran Chandler: So, what you’re kind of saying is that the use of this real-time information is pretty much overkill in a supply chain scenario because, actually, you don’t need that level of information that quickly. What’s your level of granularity do we actually need?
Joannes Vermorel: But, again, if you’re piloting picking robots in real-time in a warehouse, then 100 milliseconds might actually be too slow. So, it really depends on the type of problem. But, people who are pushing for demand sensing are not discussing the use case of real-time control of super-fast robots in automated warehouses. This is not
Kieran Chandler: What they are showing in their demos is stuff that looks very similar to the old way of doing supply chain planning, where you start looking at your sales, your demand anywhere from three weeks to nine months or even a year ahead. And that’s where I really challenge the fact that having real-time for this sort of situation is going to make any difference. That’s what I truly challenge. So who’s it really benefiting then? Would you say it’s purely a marketing gimmick to sell other pieces of software?
Joannes Vermorel: This is a case when I was thinking about this term of “moot” where you know, software you should not care about is moot. It’s literally a marketing gimmick for vendors that are short on new ideas, technologies, and probably expertise. Well, they just decide to take the road of the marketing gimmick because there is no substance whatsoever. And I know it’s relatively harsh, but frankly, I don’t see any better explanation because the problems that we’re facing are so challenging. There are so many angles that can be discussed and improved that we don’t need to invent our own buzzwords. There are so many angles that need to be refined to present something new. If you have to resort to a completely made-up buzzword that happens to be just a shallow marketing gimmick, it doesn’t put the teams behind those companies in a good light. I mean, it shows a certain indifference to the problems faced by the clients, and it also shows some degree of arrogance in the sense that they think the client is just an idiot and that you can literally push anything to them. At some point, it’s kind of an insult to the intelligence of who you’re selling to.
Kieran Chandler: So how do we differentiate them? Because there are so many of these buzzwords out there, there’s so much different research we’d have to do. How can you differentiate between what is actually a good buzzword and what is not?
Joannes Vermorel: First, I would say one way is to have a broad look at who is talking about this buzzword. Is it just your vendor, or are there plenty of people from different fields, people who do not have convergent interests, people who have no reason to copy each other? The thing about demand sensing is that basically, it was one vendor pushing it, and the other vendors just copied the shallow marketing gimmick. So you start with something that’s just a marketing gimmick, and it ended up being replicated marketing-wise by other actors. There is no substance, so it’s relatively easy to replicate. When you have actual technology, it’s harder to replicate. If the only thing you have to replicate is a fancy website, it’s much easier. But I digress.
To identify good buzzwords, look for different people talking about them. For example, something we discuss at Lokad extensively is probabilistic forecasting. If you look online for who is discussing probabilistic forecasting, you will find that there are actually tons of other communities, for example, people studying climate, heavily using probabilistic models. When you think about climate, state-of-the-art research is clearly driven by probabilistic models. It has nothing to do with supply chain directly in any way, but still, it proves that the concepts are appealing and useful to diverse groups. That, I think, is a very important flag.
Another indicator is appeal to authority. When people are just pushing for something like demand sensing as a tremendous new way of doing forecasting and make bold claims, but instead of detailing the algorithm and the math on how they do that, they put forward the name, title, and pedigree.
Kieran Chandler: Of the awful’s of the things, they all we have Dr. X PhD, why do that with 20 years of experience here and there, and that’s that’s basically when you put forward the resume of the people up front, it’s literally an appeal to authority. So that’s usually it means that you’re, you know, your ideas lack substance. Otherwise, you don’t need to have your resume in scientific papers, you just have the name of the researchers and the institution, not the resume. Okay, so if demand sensing is an example of maybe a bit of a marketing gimmick, are there lots of other examples out there in the supply chain industry of gimmicks that have kind of well historically?
Joannes Vermorel: Yes, I mean, not just supply chain. I think it’s one of those typical cases of markets that are dominated by large enterprise vendors who are playing those games a lot, but it’s not the only area. Shallow buzzwords have been quite a few. For example, IBM, 10 years ago, made a big push for their autonomous computing, which turned out to be a big pile of nothing. There were a lot of people who started to push for things like, before people were crazy about data science, they were crazy about data mining. It was a big thing 20 years ago. There was the idea of extracting data nuggets. I’m not even sure if people remember what it was supposed to be. So, there has been a long series of shallow buzzwords, and it’s actually quite hard to even remember them because precisely there is very little substance.
To prepare this episode about demand sensing was actually quite hard for me because I was reading dozens of pages about it and spending half an hour trying to summarize what I have learned. And I found myself having learned nothing. So, it’s even hard to memorize this sort of stuff.
Kieran Chandler: So, what’s normally the outcome? Do these buzzwords just fall by the wayside because there’s no substance behind them, and the industry moves on to the next one?
Joannes Vermorel: Yes, that’s my personal belief. I call it the law of preservation of hype. In physics, we have the law of preservation of mass, and as far as the marketing of enterprise vendors is concerned, there is this law of preservation of hype. There is a total mass of hype, and if you have a buzzword that enters the pile, something else exits the pile. You can actually empirically verify my law of conservation of hype by looking at Google Trends, this tool provided by Google. If you take many buzzwords like AI, cloud computing, big data, machine learning, and whatnot, you will see that over time, it’s kind of a constant, even if each buzzword has a spike and then fades. So, clearly, there are these patterns.
Kieran Chandler: Okay, so let’s start rounding things up a little bit. For supply chain practitioners looking for the next big buzzword in the future, what should they be on the lookout for?
Joannes Vermorel: I think they should be on the lookout for stuff that fundamentally addresses something pretty essential with an insight that, when looking back, is completely obvious. The surprising thing about science is that when you have things that are profoundly true and efficient, you look back at them and wonder how you could have been so ignorant before. For example, in supply chain, before we started to realize that when we wanted to look at demand, we had to look at demand with a purpose, like support. That was the birth of quantile forecasts. That was the idea that risks
Kieran Chandler: It’s obvious that optimizing stock and driving costs down are important when inventory rotates. It’s something that is so important that you cannot even unlearn it. Once you understand a core concept, it becomes very important. I would say that when you see a keyword and it clicks, it’s usually because you have understood something very fundamental. It’s not something incredibly difficult. At the core of massive innovation, it’s usually something fundamentally simple.
Joannes Vermorel: If I tell you about cloud computing, you might think it’s super complicated. But then, if I tell you it’s just hardware resources on demand, and by on demand, I mean you can just say, “Give me a computing machine in the next minute,” and there you have it. You have large-scale vendors that can sell you processing power available on demand. Once you understand that, you kind of understand a lot of things about cloud computing.
Kieran Chandler: So these keywords typically represent the core concepts, and the devil is in the details. But the key insight, if you cannot grasp it in a few minutes, might not be that valuable. It should truly strike you as a solid extra understanding for your business that you can take advantage of. Maybe to take advantage of this extra insight, you’ll need additional tools. But if there’s not a core understanding, then most probably what you’re looking at has no substance whatsoever and it’s just a big pile of nothing.
Joannes Vermorel: That’s correct.
Kieran Chandler: Great, we’ll have to wrap it up there. Thanks for your time. That’s everything for today. If you agree or disagree, make sure you drop us a comment below, and that’s everything for this week. We’ll see you next time.
