00:00:04 Introduction to software Frankensteinization.
00:00:35 Impact of software Frankensteinization on B2B software.
00:01:31 How client demands influence software Frankensteinization.
00:02:32 ‘Scars’ on software and their implications.
00:05:33 Maintenance costs of software Frankensteinization.
00:08:00 Challenges and costs of new software features.
00:08:57 Features without complexity: B2C insights.
00:10:36 Problems with B2B approach to software solutions.
00:13:18 Lokad’s experience: right vs quick & tech debt.
00:15:14 Envision: Lokad’s solution to complexity.
00:16:30 Domain-specific scripting: avoiding limitations & conflicts.
00:17:43 Addressing software bloatware in supply chains.
00:18:01 Strategies for simplifying complex software.
00:20:54 Managing ’technological mass’ in software systems.
00:22:00 IT, supply chain, marketing synergy in system changes.
00:22:43 Problems with excessive software customization.
00:23:48 Testing software vendor’s integrity via customization.
00:24:00 Gaining influence over product development strategy.
00:26:08 Choosing lean, focused software to avoid customization.
Summary
On Lokad TV, Joannes Vermorel is introducing the concept of software Frankensteinization, referring to the way B2B software evolves through haphazard modifications that don’t align with its original design. He likens these changes to ‘scars,’ contributing to the creation of a composite, evolved software system. ERPs are presented as an example, emphasizing the dilemma between maintaining software architecture and meeting new demands. Vermorel is cautioning against hasty decisions, which he says often result in software scarring and an increase in maintenance costs. While acknowledging the unavoidable complexity in software management, Vermorel underscores the importance of learning from B2C models to control feature interactions. Lokad’s response to this issue is “Envision,” a domain-specific programming language that separates infrastructural and domain-specific problems.
Extended Summary
In the latest episode of Lokad TV, the conversation revolves around the concept of software Frankensteinization, a term brought forth by Joannes Vermorel, Lokad’s founder. He uses this term to depict the transformation of B2B software over time, particularly long-lived and supply chain software, as they evolve through continuous negotiations and large deals between software vendors and their clients.
Vermorel explains that software Frankensteinization happens when adjustments are made to the software in response to client demands. These changes are often inconsistent with the original architecture, philosophy, and design of the software. As a result, the software accumulates these alterations, or “scars,” over time, turning it into what Vermorel describes as a “Frankenstein monster” - a composite and oddly evolved software system.
He further points out that this occurrence is especially pronounced in supply chain software, but it is also common in many B2B software types. Nevertheless, he clarifies that the term “scar” should not be misconstrued as inherently negative. These modifications can enhance the software by adding new features, thus improving its functionality.
Vermorel provides an example of Enterprise Resource Planning (ERP) software, which assumes that seasonality can be captured using fixed 52-week profiles. This design works well until a demand arises that does not fit into this framework, such as modeling Chinese New Year, Ramadan, or Easter, which follow different calendars and change dates annually.
In such cases, the software vendor faces a trade-off. They can cater to the new demand by integrating a “hackish” solution, which does not fit seamlessly into the software’s architecture but provides the required feature. However, this approach often leads to more “scars,” contributing to the software’s Frankensteinization.
Emphasizing the rapid pace of negotiations with key clients in the software business, Vermorel warns that the urgency can inadvertently lead to “software scarring.” He defines this as a process where hastily implemented features increase the complexity and maintenance cost of software over time.
Explaining the intricacies of software architecture, Vermorel notes that each line of code or feature needs maintenance. As more features are added, the interconnections become more complex, leading to a “quadratic complexity problem”. Essentially, potential interactions increase exponentially as the number of parts increases, causing a variety of potential issues like bugs, crashes, and security threats.
Vermorel stresses the importance of meticulous software architecture in controlling the number of interactions between features. He outlines that the goal should be to only double the amount of maintenance required if the number of features is doubled, instead of the quadrupling or more that often happens when development is rushed.
When asked how software companies can introduce new features without adding excessive complexity, Vermorel suggests learning from B2C software companies like Google and Netflix. They take time to understand the specific problems they’re trying to solve and design solutions that address a broad category of similar problems. He contrasts this approach with the B2B world, where customers often come with not only problems but also their own proposed solutions.
Regarding how Lokad manages these issues, Vermorel reveals their initial struggle in balancing between doing things correctly and quickly accommodating customer requests. They noticed increasing technological debt in the early years, which Vermorel acknowledges as detrimental. They realized that while adding features to please individual clients might seem beneficial in the short-term, it leads to a complex, inconsistent, and hard-to-manage product in the long-term.
In reaction to this realization, Lokad devised a unique solution: a domain-specific programming language called “Envision.” The aim of Envision is to separate infrastructural problems from domain-specific problems, enabling Lokad to manage the addition and maintenance of new features more effectively without adding excessive complexity.
Vermorel continues to elaborate on how software products, especially in the supply chain industry, frequently deal with a bloating issue due to extensive customization for each client. While customization aims to deliver bespoke solutions, it often results in a complex, hard-to-handle system requiring significant effort to maintain and upgrade. To avoid this, Vermorel shares how Lokad uses a domain-specific script, Envision, to create highly customized, yet manageable solutions for their clients.
Addressing the topic of software complexity, Chandler inquires about what software companies can do to prevent bloatware. Vermorel responds that the scenario depends on whether a company has access to its software. If not, the company needs to cope with the complexity of the acquired software. However, if access is available, the software can be simplified by identifying and removing unused elements, greatly reducing complexity.
Switching the focus to the responsibilities of IT and supply chain executives, Vermorel observes that while IT personnel usually have the technical know-how, they often lack an understanding of the business value of certain features. This leads to a disconnect between necessary and unnecessary features. Therefore, business decision-makers must collaborate closely with IT to drive prioritization and system simplification.
Vermorel provides advice for supply chain executives looking to invest in new software. He cautions against demanding excessive customization, as it can bring about more issues. Instead, executives should maintain regular contact with the vendor’s product managers or CTOs to communicate their needs directly. Rather than forcing specific features into a contract, the aim should be to present problems to those capable of engineering solutions.
Finally, Vermorel advises businesses to choose software that excels at one thing, instead of opting for comprehensive solutions. This focused approach facilitates improvements and reduces complexity caused by a plethora of interconnected features. By staying close to those who control the roadmap and choosing sharply focused software, businesses can better manage complexity and enhance productivity.
Full Transcript
Kieran Chandler: Today, we’re going to be tackling the subject of software Frankensteinization. So, Joannes, this is a topic that’s fairly difficult to pronounce, so I’m probably going to leave this one to you. I’m assuming we’re not talking about a big green monster today. What exactly are we talking about?
Joannes Vermorel: We are talking about a process that drives the evolution of software, specifically B2B software, and even more specifically, supply chain software. It mostly applies to long-lived B2B software. This process, which I refer to as “Frankensteinization,” evolves the software over the years with an ongoing stream of large deals negotiated between the software vendors and its clients. The interesting thing, where the “Frankensteinization” part comes from, is that every large deal negotiated between the vendor and one of its large clients is likely to leave a scar in the software.
It’s a gradual process where a vendor negotiates with a large company. The large company has demands, and in order to meet those demands, adjustments are made to the software that don’t fit with the overall architecture, philosophy, or original design. The feature gets added, but in a somewhat hackish way. That’s a scar. The problem is, if you repeat this process over and over for years, what you end up with in terms of software is, in a way, a Frankenstein monster. It’s a beast made of many scars that look somewhat hideous and super composite. It evolves in weird ways, and that’s how you end up with these Frankenstein software monsters. This is something that happens in most databases, but in supply chain software, it’s like it’s on steroids.
Kieran Chandler: Let’s talk about these scars, as you refer to them. What do they look like? I mean, they’re improving the software, right? They’re adding extra features. Is that such a bad thing?
Joannes Vermorel: Yes, obviously that’s the trade-off. You want to improve your software, but a piece of software is a very complicated object. It’s not like a building where you can just add a floor or expand nicely. Software comes with lots of inviolable design assumptions made very early on that give the software some integrity—architectural integrity.
Let’s take many ERPs as an example, which were built around the idea of capturing seasonality with profiles that are exactly 52 weeks, representing a given year. So, you literally have a table with 52 columns, one column per week, and you have all these seasonality profiles that you can apply to any item you’re producing or selling. But what if you want to model something like Chinese New Year? It doesn’t fit into this 52 weeks grid because it’s going to move from one year to the next according to the traditional Chinese calendar, not the Gregorian calendar. You will face similar problems with Ramadan or even Easter.
You have this very nice assumption that can fall apart when you face a demand that doesn’t fit into your framework. And that can take many forms. The problem is that as a software vendor, you don’t actually have the time to rewrite your entire software stack so that these new features integrate into your architecture in a completely native way. In the end, it’s a bit hackish.
Kieran Chandler: You’re negotiating with an important client, so you don’t have years to close the deal. And then, you don’t have years to deliver the future either. So, things have to be done in a relative emergency every single time, just due to the fact that it’s actually a bigger than usual negotiation to close a sale. Things are rushed almost by design. But when does something become a scar and when does something become a good feature? Because all these software’s have got to evolve, they’ve got to change, they’ve got to work for their clients. So what differentiates the two? How do you know that something you’re developing isn’t going to become a scar?
Joannes Vermorel: The question is how much will it cost in terms of maintenance? Every single line of code that exists in your big enterprise software is something that you will have to maintain. One thing that people do not always realize, even software professionals, is that the software is like intricate machinery where every single part has to interact somehow with all the other parts. You have a quadratic complexity problem.
What does that mean? It means that if I have ten parts, I’ve basically got about 100 potential interactions, you know, 10 x 10. If I have a thousand parts and they are all interacting, then I have 1,000 by 1,000 interactions. Every potential interaction is a recipe for all sorts of problems: security issues, bugs, crashes, incorrect results, or just massive slowdowns in the software.
When you add more parts, you increase the number of potential interactions between the parts in your software, and it increases much more rapidly than the number of parts. And when I say parts, you can think of features, capabilities, screens, data options, or entries.
If you are very careful with your software architecture, you can preserve the number of interactions between your features to keep it under control. If you double the number of features in your software, you don’t want to multiply the number of interactions by 4, because that means when you double the number of features, you actually quadruple the amount of maintenance you need to do.
But it’s very difficult. When you’re rushed, when you double the number of features, you actually multiply by 4 or even more the amount of effort that you have to invest in maintenance. Over time, the maintenance cost skyrockets, your capacity to roll out new stuff gets lower and lower. Whenever you introduce a new feature, it triggers an entire wave of incompatibilities and bizarre unintended interactions between different parts, because it has not been completely thought through. The pain increases over time.
Kieran Chandler: Okay, so let’s think of things from that software company’s perspective. How can they introduce these new features? How can they implement these features to keep their clients happy, without introducing this extra layer of complexity? What can they do?
Joannes Vermorel: I think the lesson comes from the world of B2C software. Google doesn’t roll out new features for Google Search, or Netflix doesn’t roll out new features for Netflix just because they are signing up a new client. They don’t take on new clients and start discussing with them and saying, “Oh, if we don’t have those features, then you will not sign up with us, so we need to do it now.”
They don’t work like that. They think of what is the domain, what is the specific problem they’re trying to solve. They try to have a very high perspective on this problem and then think.
Kieran Chandler: You discuss an approach that aims to generate solutions to an entire category of problems, rather than a single micro problem. This necessitates re-architecting software to address a broad spectrum of similar problems. And this happens all the time. When interacting with clients, you listen to their problems, not their proposed solutions. In comparison, in the B2B world, customers tend to present not just their problem, but also a solution. This solution may or may not fit into your existing software stack or the evolution of this problem. So, is it about not really listening?
Joannes Vermorel: Precisely, it’s more about really understanding the core problem instead of fixating on a particular solution. Big companies like Google and Netflix are excellent examples of this.
Kieran Chandler: So, you’re saying that companies aren’t really listening. But what about those annoying pop-ups asking for feedback? Is anyone actually looking at those?
Joannes Vermorel: The pop-ups you’re referring to are typically undesirable features. But credit must be given to companies like Google. These pop-ups, where you have to comply with the terms, were not their choice. They were forced to implement them due to regulatory constraints. So, while it may seem like a bizarre feature, it wasn’t their doing. Also, if you look at who won the web, it wasn’t the companies with super annoying ads. It was those like Google who had clean, unobtrusive ads. They were thinking of their customers instead of devaluing their product with something so annoying. They take the time to think their features through.
Kieran Chandler: In B2B software companies, particularly in supply chain, there’s a lot of diversity. There can be hundreds of scenarios, and often at the last minute, people negotiate lots of features that almost always turn out to be bad ideas six months later. So, would you agree that software Frankensteinization is a bad thing? And if so, what have you done differently at Lokad to avoid this problem?
Joannes Vermorel: Absolutely, software Frankensteinization is a problem. During the first few years at Lokad, we faced this very issue. It was a tough trade-off. If you want to do it right, it can take one or two years, which is too slow. If you don’t do it right, you can wrap up in a few weeks or months, but then you’re left with a big scar, an ugly hack. You end up accumulating technical debt. So, for the first few years of Lokad, we didn’t have a solution, but we were becoming increasingly aware of the problem. Even as a startup, the technological debt was mounting, which was a concerning situation. It was at this point that I realized the trend was unfavorable. Looking a couple of decades down the road, I could see how competitors, with their bloated software for supply chain optimization, were just adding more and more features without any consistency.
Kieran Chandler: It seems that software companies are always adding new features, and the end result can be a confusing, bloated product. Can you talk about your approach to this problem at Lokad?
Joannes Vermorel: Absolutely. The way software tends to become bloated is by adding one bad feature at a time. If you continue on this path for 20 years, you end up with a monstrous product. It’s not that the developers are incompetent or foolish, they are simply doing a decent job incrementally, trying to win one client at a time. However, this approach scours the software one client at a time, and the end result is typically very, very bad.
So, we decided to take a completely different angle at Lokad, and that led to the birth of Envision, our domain-specific programming language. With Envision, we effectively split all the problems. We separated the infrastructure, the data infrastructure, and machine learning infrastructure. These are long-living products that we want to maintain and upgrade, and it’s a multi-year effort every time we decide to change and upgrade them.
Then, separately, for every single client, we create a completely bespoke implementation written in scripts in Envision. As Envision is specifically designed for supply chain optimization, we could write something completely custom for every single client, with high productivity, while making it completely bespoke.
So, we start with a blank sheet, customize it completely, and then we don’t have to face a situation where the client asks us for something that we don’t support. Because of Envision’s programmatic capabilities, if we have to do something special for a client, the worst-case scenario is that the script we write won’t be as compact as usual. However, we can still benefit from an infrastructure designed to make it easy to solve certain types of problems.
If you introduce advanced programmatic domain-specific capabilities into your platform, then suddenly you don’t have to rush a negotiation that will scar your product. You can do the customization in your programmatic environment that is bespoke for every single client, and then keep rolling out upgrades for your platform on your own schedule, which will most likely not match the schedule of your clients.
Kieran Chandler: You identified this issue quite early on. What advice would you give to other software companies experiencing this kind of bloatware? Can they do something about it? Should they start removing some of these features to simplify things?
Joannes Vermorel: The situation depends on several factors. If you’re a large company that owns the software, such as a WMS, ERP, MRP, etc., you may not even have access to the software itself, it could just be a vendor product. If you don’t have access to the product, then you’re stuck with the complexity of the software that you’ve acquired.
If you do have access to the software and want to simplify it, then yes, you should start by looking at all the things that you’re not using and aggressively discard them. Many people are typically worried about removing parts of an existing piece of software because they’re concerned about losing certain capabilities. It’s true that if you remove capabilities, you lose those capabilities. However, if you remove something that’s almost never used, you can also remove entire classes of problems that were caused by the very presence of this tiny feature.
We often see ERP implementations at Lokad with literally thousands of tables. You can have one ERP with, let’s say, five thousand tables. Each table can have anywhere from 10 to 200 fields, resulting in massive complexity. Even just backing up the ERP can be incredibly complicated.
Kieran Chandler: Because you have so many tables, you need a table to store the list of the tables and whatnot. That’s the sort of situation where if you can identify tons of tables that are not even used anymore, or screens that are not used anymore, by just removing them, you simplify everything.
Joannes Vermorel: Exactly, for example, if you have to upgrade from one version of your database to another, one of the issues that you can encounter is that you need to upgrade pieces of logic that are not even used by anyone anymore. Every single line of code that exists, be it Java, C Sharp, SQL or whatnot, is something that you need to maintain for as long as the line of code exists. If you remove that, it means during your next integration or migration, people will not have to ask the question of how do we migrate this thing to the next version of the system or to the next system altogether.
And this is something that supply chain executives need to think about. When purchasing software, they should really adhere to the principle of parsimony - if you don’t really need it, it’s better not to have that part of the system. It will just be a burden. It’s vital to constantly pay attention to the technological mass of the solution that you’re acquiring and operating.
This technological mass does not come free. If you have more screens, you need more people to train them. If you can remove a couple of screens, that would mean less training, fewer bugs being reported, fewer fights with IT and so on. It’s about managing the complexity of the IT, but the tricky part is that typically, IT has no clue about that because they do not have access to the business value. They cannot differentiate between features that are really necessary and features that are not absolutely necessary. They can’t translate tables, fields, screens, or capabilities into euros or dollars. That’s something that only people on the supply chain or marketing, if you’re looking at a different system, can do. That’s why IT needs the backup of those people, the decision-makers in the company, to drive the prioritization and change at the simplification of the system.
Kieran Chandler: And so, to wrap things up, you mentioned that if I’m a supply chain executive that wants to invest in some new software for my company, it sounds like I shouldn’t really be asking for that much customization because that introduces problems. So, what should I be looking for?
Joannes Vermorel: Yes, asking for customization or special features is like a recipe for generating problems. If your software vendor didn’t have enough problems of its own, you’re just creating new ones that will just make things worse. So really, you should not do that. If you can actually ask for customization, just as a test, because if the vendor willingly engages in the discussion with you about customization, it means that the vendor has no integrity. That means the vendor does that with every single other client, which means that even if the product is good today, in five years from now, it will be a nightmare.
Firstly, if you ask for customization, it’s good to ask and you should really expect that this request should be denied. If not, then that’s a problem.
Kieran Chandler: One thing that people should ask for is special features, correct? I mean, if there’s one thing to be asked for, it might be access to the CTO or the product manager on the vendor side. Include it as part of the contract, perhaps? Like wanting to have a meeting with this person two hours once a month. Why?
Joannes Vermorel: If you negotiate to have direct access to the people who are engineering the roadmap of the vendor, you can just present your changes. You don’t need to impose your solution on them; just present your problem. Because, as engineers, what do they do when presented with a problem? They start working on a solution. If you have a routine meeting where you present your problems, you don’t need to negotiate for any specific features in the contract. In a couple of years, features will start to appear in the product that seem to fit the very problem you’ve exposed.
Kieran Chandler: Can you explain a bit more about that?
Joannes Vermorel: If you do the math, consider the CTO of a software company. How many meetings can this person have with clients in a given month? Let’s say they have a maximum of 20. If you have one meeting a month, you’re essentially getting around five percent of the mental bandwidth of the CTO, who is driving the technological development of the vendor you’re working with. This means your problems are getting a significant amount of attention.
Kieran Chandler: So what’s your suggestion?
Joannes Vermorel: My suggestion, if you want to play it smart, is to get close to the people who have control over the roadmap. It’s smarter than hastily negotiating for broken features that you’ll likely discard in six months because your solution wasn’t good or your problem has changed. Also, a different class of advice would be to avoid software that does too much. You want to think of a piece of software that does one thing and does it well, rather than a software with extensive capabilities that ends up doing everything poorly.
Kieran Chandler: Can you elaborate more on that point?
Joannes Vermorel: It’s easier to improve a tightly focused piece of software than to improve a gigantic framework that is doing everything. Whenever you’re going to adjust something, think of it as the quadratic number of interactions. If you have a thousand features and you add one, you need to think of all the interactions with the pre-existing 1,000 features. So, if you introduce one feature, you look at all the 1,000 interactions. However, if you have only ten features and you add an 11th, it’s only ten interactions to review. So again, focus on something that is lean and sharply focused on the specific problem you’re trying to solve.
Kieran Chandler: Great! I’m afraid we’re going to have to leave it there for today, but I’m guessing there’s a few CEOs out there who might not thank you too much for that, likely getting a few more phone calls now. Well, that’s everything for this week. We’ll be back again next week with another episode. Until then, take care.
Joannes Vermorel: Thank you.
