00:00:03 Modularization in IT and supply chains intro.
00:00:26 Physical supply chain modularity.
00:02:31 Early IT systems’ monolithic structures.
00:04:08 IT context of monolithic systems.
00:05:31 Internet’s impact on modular software design.
00:08:02 Problems in monolithic software systems.
00:08:48 Inefficiencies of monolithic systems.
00:11:16 Transitioning to modular systems: examples.
00:13:34 System failures in modular applications.
00:14:34 Redundancy, uptime in modular systems.
00:16:00 Large system reliability, software project failures.
00:16:50 Lidl’s failed software transition example.
00:18:09 “Fail fast” concept in software projects.
00:19:30 Managing overlaps, importance of “glue” software.
00:22:46 Avoiding software disasters.
Summary
In the Lokad TV episode, Joannes Vermorel, the founder of Lokad, is discussing the significance of modularization in business IT infrastructure, drawing a comparison with the adaptability of supply chains. He refers to the monolithic IT systems from the ’70s and ’80s, characterized by their inflexibility due to technological limitations. Even with the shift towards networked systems following the internet’s emergence, he underscores the ongoing challenge for businesses to modularize their IT systems. Vermorel proposes Service Oriented Architecture (SOA) as a potential solution, stressing the necessity for smaller, well-defined services. He cautions against undertaking large-scale projects and advocates a “fail fast” approach, emphasizing the criticality of risk management and rapid recovery in the event of failures.
Extended Summary
In the current episode of Lokad TV, host Kieran Chandler engages Joannes Vermorel, the founder of Lokad, in a discussion about modularization in business IT infrastructure. Vermorel outlines the concept of modularization, drawing parallels primarily with supply chains in the physical world, which he considers as humanity’s most modular creations.
He uses the examples of trucks, pallets, and containers to clarify his points on modularity. Vermorel emphasizes the inherent modularity of physical supply chains by pointing out the universal applicability of barcodes, which can be attached to virtually anything.
Although physical supply chains display clear modularity, Vermorel observes that company IT infrastructures often lack similar flexibility. He traces this lack of modularity back to the early stages of IT development within companies during the 1970s and 1980s, when initial IT implementations and ERP systems were designed as standalone, monolithic structures due to the technological limitations of the time.
Vermorel identifies a ‘monolith’ in IT terms as a unified application that cannot be broken down into smaller parts. He explains how this monolithic approach contrasts with the modularity seen in physical supply chains, where various components can be combined or separated as needed.
Despite their complexity, Vermorel explains that the early mainframes were inherently coherent as they functioned as a single unit with all operations centralized. He notes how this form of system design seems outdated to the current generation accustomed to distributed systems and network-based applications, marking a significant shift in IT infrastructure development over the years.
Vermorel points out that the landscape started to change with the advent of the internet in the late 90s, leading to the design of software with isolated parts connected by a network. However, he remarks that many companies still struggle with modularization, despite the value of modular IT systems being well understood in theory.
He explains how many vendors have adapted these old monolithic architectures to cloud and Software as a Service (SaaS) models, resulting in systems that offer improved maintenance but do not enhance modularity. He says this lack of modularity impedes companies from isolating and changing specific features.
To overcome this issue, Vermorel proposes a Service Oriented Architecture (SOA), an approach that breaks down capabilities into smaller, well-defined “chunks.” He references Amazon as an example of a company that successfully embraced this modular approach early on.
He advocates for a service-oriented architecture with APIs that enable data to be exported and exploited across different divisions of a company. Despite acknowledging that a decentralized approach presents potential for more points of failure compared to a monolithic system, Vermorel argues that these challenges can be mitigated through redundancy and high-quality engineering.
Vermorel estimates that between a third and half of all supply chain IT projects fail. He references the case of Lidl in Germany, which lost half a billion euros on a failed SAP transition. He argues that transitioning from small vendors to large vendors only slightly reduces failure rates, but doesn’t eliminate them.
Regarding the management of multiple applications, Vermorel suggests choosing applications with a narrow scope to minimize overlaps and advises against buying large frameworks from vendors. To effectively manage multiple apps, he says companies should develop internal “glue” that binds these applications together, facilitated by a service-oriented architecture.
Lastly, Vermorel shares advice on how to prevent disasters like Lidl’s failed SAP transition. He emphasizes thinking in terms of risk management and adopting a “fail fast” approach. He advises against wishful thinking in project management and underscores the importance of planning for potential failures and finding ways to recover quickly.
Full Transcript
Kieran Chandler: Hello and welcome back to another episode of Lokad TV. This week we’re going to be discussing modularization, the process of subdividing our company’s computer processes from one stand-alone system to a number of other sub-programs or even apps. As always, I’m joined by Joannes Vermorel. So, Joannes, what part of the business’s processes are we talking about making more modular today?
Joannes Vermorel: Today we are focusing especially on the IT infrastructure and the applicative landscape of your company. Modularization is interesting. When you think about the physical world, it’s evident that supply chains are incredibly modular. They are probably the most modular things ever invented by humanity. By modular, I mean that if you want to transport goods by roads, you can use trucks. Trucks are incredibly modular because you can put pretty much anything that does not exceed the capacity of a truck in it, and a truck can drive on any road, going from anywhere to anywhere else. If you need extra trucks, you could pretty much use any kind of truck, almost, and they can add transportation capacity to your supply chain.
The same thing applies to pallets and containers. You can put almost anything on a pallet or in a container as long as it doesn’t exceed capacity. A container is extremely modular. You can move it from a cargo ship to a truck. All those pieces are incredibly modular. You can put a barcode on pretty much anything that is not too small. It’s an incredibly modern invention. If you have something like a diamond, you would probably put it in a box and then put the barcode on top of the box. You can combine all these simple elements at will, nearly infinitely. It’s very Lego-like; simple parts that you can combine in incredibly varied ways. That’s the physical supply chain, which is incredibly modular.
However, the interesting thing is that when you move to the IT world, we enter a perspective that is completely different and where frequently modularization is lost. I think that the origin of that dates from the early 70s or 80s where companies started to have their first computer systems and their first ERP implementations.
Kieran Chandler: So what you’re saying is that those initial implementations, those initial IT systems, because they were very much sort of a singular approach, that’s still what we’re using today. Is that what you’re saying?
Joannes Vermorel: Yes, indeed. When you think back at the IT or software of the 70s, 80s or even the early 90s, doing anything over the network was like rocket science. It was possible, but it was an engineering nightmare. It was much easier to say we take one big machine, ideally an IBM mainframe of the time, and put all your logic on this machine. Then, everybody is using a terminal connected to this machine, and all the logic is running in a monolith on this machine.
Kieran Chandler: So what do you mean by monolith?
Joannes Vermorel: By a monolith, I mean that it’s like an app that is a whole. It cannot be taken apart. It has to be together or it’s nothing.
Kieran Chandler: I’m afraid I’m a bit of a millennial, so this kind of idea might predate me a little bit. But basically, what we’re saying is that everything is being connected to one single machine, correct? And then we’re all connecting and we’re all working from there?
Joannes Vermorel: Exactly. Mainframes were relatively complex hardware things, so even if it was one machine.
Kieran Chandler: We’re mostly talking about ERPs, that is, Enterprise Resource Management systems. These systems are typically designed to be extensible, allowing the addition of extra capabilities and features. However, they aren’t modular, meaning you cannot separate those capabilities to keep them fully isolated.
Joannes Vermorel: The interesting thing is what really changed was probably the Internet. I’m not referring to the invention of the Internet, but to the fact that during the late ’90s, with the Internet becoming very popular, people started to think about how you can design software so that you take the parts in isolation, with the network in the middle. This way, the workflow is not an engineering nightmare. Before this, if you did not know how to build a complex software system made of many parts, having a network in the middle was an engineering nightmare. This know-how and culture and tools mostly emerged as a byproduct of the adoption of the Internet by the general public.
Kieran Chandler: The Internet’s been around for a long time now, so why are we still talking about modularization? Why are these software still kind of acting in this kind of single manner?
Joannes Vermorel: Right now, my diagnosis is that when you take the average company that is operating a complex supply chain or multinational with many locations, everything physical is incredibly modular. However, when we start looking at the IT, it’s completely monolithic. I believe that companies and the market at large are still having trouble appreciating the value of having something that is extremely modular in terms of IT. Physically, it’s pretty obvious, and supply chains are still improving their modularity. However, in terms of the application landscape, it’s more abstract, more difficult to perceive the modularity as such.
Many vendors took old monolithic architectures, slightly upgraded them towards SaaS and the cloud, but just copied and pasted. It’s fundamentally the same sort of architecture that you had in an IBM mainframe in the ’90s, where you just decided that instead of having a machine at your company’s headquarters that runs the thing, you delegate it to a software vendor that operates as SaaS. But if the SaaS vendor only has a monolith that they run on a machine that’s just far from your headquarters, with just a bit of networking and a web user interface in the middle, it doesn’t add anything as far as modernization is concerned. It just makes maintenance easier. But when you want to take the features apart, you’re still faced with a monolith where this sort of capabilities cannot happen.
Kieran Chandler: What’s the problem with this monolithic approach? How’s it affecting companies in the real world?
Joannes Vermorel: Just imagine the physical equivalent of lack of modularization. Imagine that in your supply chain, whenever you want to change one truck, you have to change them all. For example, if you change from one truck to another, you need a different type of gasoline. So, all your gas stations would need to be changed, meaning that you have reservoirs that contain gas where you need a second type of gas. That would be immense.
The pain, you see, is comparable to the situation in software. When you lack modernization, it’s like if you change one part, you need to change them all. If you change one truck, you need to change all your fleet. Imagine if you change the shelves of one warehouse and you would need to change all the shelves of all the other warehouses, otherwise, it’s not compatible. Then you realize, okay, maybe I can decide to upgrade my fleet of trucks toward electric trucks, but that’s going to be a massive strategic move and it’s very complicated. You would still prefer to have something relatively modular where you can have electric trucks and combustion trucks coexisting gracefully for a long period of time. When you lack modernization, it means that you cannot have this sort of coexistence. You cannot change some bits of your supply chain without changing everything.
The most frequent anecdotal evidence of this lack of modernization is that for companies to move from one warehouse to another physically, it could be done in like a day where you just move the stuff that is sitting in a warehouse from the shelves of warehouse A to the shelves of warehouse B. But to migrate the software that was connected to warehouse A so that everything knows that all the stuff is in the system that drives warehouse B, that would take like six months.
Kieran Chandler: That’s interesting. How do big companies do this if they’re existing on this monolithic kind of system? How do they migrate to a more modular system?
Joannes Vermorel: I think we are going to go back to one of the companies that we mentioned in pretty much every single episode, like Amazon. They’re not the only one who did adopt an extremely modular approach. In Germany, Zalando, you can follow on the blogs, have now also fully adapted a very modular approach and the keyword in IT for this when you want to have this modularity is probably Service Oriented Architecture, SOA.
Service Oriented Architecture basically means that you want to isolate capabilities in chunks that are themselves like small monoliths, but much smaller, and that they are very well scoped to do one thing and do it well. And you plug all those things together through your Service Oriented Architecture which means that every single service, in the sense it’s an app that does one thing and does it well, exposes APIs so that it can be plugged into your applications landscape very easily. It’s designed to be easily pluggable programmatically into any arbitrary applications landscape without making almost any assumptions on what are the other parts of the applications landscape.
Amazon’s Jeff Bezos published a very famous memo, I believe in 2002, where to all his teams he said that every single division needs to have service-oriented architecture with APIs, so that the data that you have in your silo can be exported to be exploited in any other division in the company and we can interact programmatically with whatever you’re building.
Kieran Chandler: The problem I see with this is that you’re becoming reliant on a lot of smaller apps, smaller programs, smaller companies ultimately. Doesn’t that introduce a lot more single points of failure? Whereas if you’re having a more monolithic approach, you’ve got one solid large company, one large ERP system that will always be running and you’ve got more confidence in it.
Joannes Vermorel: It’s true, and to some extent that’s one of the challenges of a distributed system. If your hardware fails 1% of the time, then that means if you have a mainframe.
Kieran Chandler: You’ve mentioned that the software can break down three times a year. If you have 10 apps each with a one percent chance of being down on any given day, it means roughly every 10 days something is down in your network. It’s indeed a challenge. What solutions would you suggest for this situation?
Joannes Vermorel: Redundancy is the first thing that comes to mind, but I’d like to discuss the implications in terms of distributed computing, and maybe then we can talk about small versus large companies and the disparity within vendors. From a distributed computing viewpoint, the goal is to ensure that every single block is highly redundant. This is the essence of cloud computing. You want to have heavily redundant software so that your uptime is very high.
The good news is that because your apps are smaller and simpler, it’s much easier to achieve very high uptime with a small simple app than it is with a very complex app. There are many component services on the internet that are literally always on. Google Mail, for instance, is literally always on. Yahoo’s website is basically always on. Facebook is also always on. So, it’s possible to engineer these ‘always-on’ properties for your apps, which addresses your system’s reliability as a whole.
Kieran Chandler: What about transitioning from one large vendor to many small vendors?
Joannes Vermorel: The reality is that failure rates in software are high. People fail to realize that maybe half of the time, software initiatives fail. My estimate would be that between a third and a half of all supply chain IT projects fail for companies of pretty much any size. A few weeks ago, Lidl in Germany basically busted half a billion euros on a failed SAP transition. It was a seven-year project that ended in failure, and they ultimately gave up. But it’s not the only example; this happens quite frequently.
Large vendors probably have a better success rate, but if we’re talking about ballpark figures, we’re talking about going from a small vendor with a 50% failure rate, which is pretty bad, to a big vendor with a 30% failure rate. So when you transition from a small to a large company, your risk diminishes slightly, but only a little.
If you opt for a highly modular approach, yes, you will have many things that will fail, but you will have the opportunity to try again and repeat until you have success. Each component has a chance to fail, but because the scope is simpler, the app itself is smaller, you can fail fast and retry.
The thing with the Lidl approach is that I’m pretty sure they initially weren’t planning a seven-year migration. It was probably a two-year migration that turned into three, then four, and so on because they were failing, reiterating, and refailing. Seven years down the line, they finally decided to give up, probably because everyone had lost any hope for the project to succeed.
Kieran Chandler: So if we agree that these apps, once you find the right one and perhaps iterate a bit, can be reliable… They seem to work and they seem to be doing the same job as a larger sort of system. What about the crossover between those apps? Because a lot of the time, an app would do something else well and there might be a little bit of crossover and conflict within that system. How do you sort of manage that?
Joannes Vermorel: First, when you choose the composition of your applicative landscape, you really want to choose apps that have a narrow scope. This approach goes completely against what most Request for Proposals (RFPs) do. When companies are seeking to acquire a piece of software, they often go for an RFP and want everything – all features, all bells and whistles. But that’s the opposite of what you should seek. You want something that is extremely narrow, so that by design you minimize your overlaps.
If you’re buying large frameworks, it’s a recipe to have overlaps. Enterprise vendors are interested in selling you large frameworks because they can extend it with many features. They sell you something, and then they will sell you the add-ons on top of it. So, I would say to people running large supply chains, be very careful when you’re being sold a platform.
A platform is good, but two platforms can be a nightmare. As soon as you have several vendors selling you platforms, you will end up with a sea of overlaps. The solution is carefully choosing the composition of your ingredients.
In terms of “glue”, the approach is that it’s typically something you want to develop internally. This might go against the intuition of why you would want to develop any kind of software in-house. The answer is that if you’re a company of a certain size, your applicative landscape is completely unique to you.
Even if you use all the standard apps, you will need multiple apps. There is no way that you will have an ERP nowadays that can do emails, video conferencing and whatnot. You cannot push every single software feature that you need into one app. You will probably use Microsoft Excel like everybody else, so you will need to have a place to store files and so on.
It’s not realistic to say we have one piece of software. Any company of significant size is going to be a collection of software anyway. The exact recipe, that is the list of software that literally runs your company, is going to be completely unique to you anyway. No other company is organized exactly like you.
Kieran Chandler: It’s completely unique, that’s your DNA. You can have this glue that you implement in-house. That’s the whole point of the service-oriented architecture, to make this glue as simple and lean to implement, so that you can have a very lean IT team that has minimal efforts to do. To come up with this custom software that just glues things together. So, to wrap things up, if I’m a CEO, how do I avoid another Lidl-type disaster?
Joannes Vermorel: First, I think it’s essential to think risk instead of minimizing the risk. The Lidl-type disaster was people saying, “We want to take no risk whatsoever”, and so they go for the biggest vendor out there and try to have the product that does everything. Essentially, it’s the approach where we say we want something that is correct by design, and we want to roll out something that upgrades the whole company at once. This is the worst type of approach in terms of risk management.
You need to think completely the opposite, which is, “How can I have something that will fail fast if it has to fail?” See, the problem with the risk is that when you have this massive project, you do not know for a long period whether you’ve failed or not. What you want is to have something that is more “fail fast,” where you know if you failed, and the failure is on a small scale. And if you need to have a replacement, you have a scope that is manageable, and you do that with many small chunks.
So, think of it as opposed to preventing the risk. I’m saying that going from a large vendor to a small vendor, you might increase the risk from 50% chance of failure to 50%. In the end, if a 30% chance of failure means that your company goes bankrupt or that the supply chain grinds to a halt, it’s not a risk that you can take. So, you have to plan for failure anyway.
My thinking is that because a high degree of risk is unavoidable, go directly for a plan for failure. Try to have small, fast, and well-scoped failures so that you can iterate, as opposed to saying we are going to do everything right the first time, which is wishful thinking. Then, embark on a journey that can likely last seven years, to finally lose half a billion euros in the process. That’s the kind of wishful thinking about project management.
Kieran Chandler: Great, I like it. The Lokad tip of the day: If you’re going to fail, fail fast. Brilliant. So that’s everything for this week. We’ll be back again next week with yet another episode. Until then, thanks for watching.
