00:00:08 Domain-specific languages (DSLs) and their diversity.
00:01:36 Examples of DSLs: AMPL for mathematical programming and HTML for web pages.
00:03:00 The advantages of using DSLs over general-purpose programming languages for specific tasks.
00:05:22 Lokad’s use of a DSL for supply chain optimization in diverse situations.
00:07:23 The benefits of using a DSL for faster and more reliable configuration.
00:09:35 Limitations of a DSL and their benefits in supply chain optimization.
00:12:55 Security implications of using a DSL compared to general programming languages.
00:13:01 The time-consuming process of developing a DSL.
00:14:49 Advice for startups considering developing their own DSL.
00:16:01 Underexploitation of DSLs and traditional solutions leading to unsatisfying products.
00:17:26 The dilemma of using DSLs versus mainstream programming languages and their outcomes.
00:18:32 SQL and HTML as successful examples of domain-specific languages.
00:19:46 Potential of DSLs in various industries and their transformative power.
00:21:19 Conclusion and potential future developments in DSL usage.
Summary
In this interview, Kieran Chandler discusses domain-specific languages (DSLs) with Joannes Vermorel, the founder of supply chain optimization company Lokad. DSLs are programming languages designed for specific tasks, unlike general-purpose languages like Java or Python. Vermorel explains that DSLs are used in industries such as automotive, where they provide bug-free operation for critical components like anti-lock braking systems. He also discusses Lokad’s development of their own DSL, Envision, to streamline supply chain optimization. Vermorel highlights the advantages of DSLs, such as efficient calculations and customization for specific domains, while acknowledging the challenges and time-consuming nature of developing them.
Extended Summary
In this interview, Kieran Chandler, the host, discusses domain-specific languages (DSLs) with Joannes Vermorel, the founder of Lokad, a software company specializing in supply chain optimization.
DSLs are a specific class of programming languages designed to solve very particular types of problems, as opposed to general programming languages like Java or Python, which are intended to address a broader range of tasks. General programming languages are designed to provide a convenient, productive, and efficient way to do everything that can be done with computers, while DSLs focus on solving specific problems. Some of the first domain-specific languages, such as AMPL, were created for mathematical programming and other very specific tasks.
DSLs are used in real-world applications to implement critical components in various industries. For example, in the automotive sector, a DSL might be used to develop the software controlling the anti-lock braking system (ABS) in a car. The goal is to create a near bug-free system, as a failure in this component could lead to a loss of braking capability. In contrast, mainstream programming languages might not provide the same level of certainty in bug-free operation.
Another example of a DSL is HTML, which is designed for creating web pages. HTML is more accessible and simpler than general programming languages, making it suitable for non-technical users or even primary school students. However, this simplicity comes with limitations, as HTML is constrained in its capabilities, only allowing users to control the layout of a web page rather than enabling more complex tasks like controlling robots or developing artificial intelligence.
When asked why DSLs are used instead of more expressive mainstream programming languages, Vermorel explains that using a general programming language for tasks better suited to a DSL would overcomplicate the process. For example, using a general programming language for web page design would mean controlling every single pixel on the screen, instead of using simple markup language elements, like specifying title, font size, or text justification, as is the case with HTML.
Vermorel explains that supply chain problems are diverse, with no two companies being exactly alike. Companies have differing IT landscapes, applicative landscapes, and various combinations of ERPs, WMSs, and other systems. This diversity creates challenges for software solutions that attempt to address supply chain optimization through a one-size-fits-all approach. Traditional frameworks that rely on configuration often require extensive customization and configuration for each client, which is time-consuming and costly.
Acknowledging that every client situation requires different software, Lokad sought to develop a more efficient and reliable process. They decided to create their own DSL, Envision, to streamline the configuration process and make it faster and more productive. Vermorel contrasts the use of Envision with mainstream programming languages like C#, F#, and TypeScript. While they are already extensively using these generic programming languages, tackling client requests with them was found to be sluggish and costly. Envision was designed to be more agile and better suited to the unique demands of supply chain optimization.
Vermorel highlights that one reason many supply chain software solutions become bloated is because they try to accommodate every possible feature and use case. Lokad chose a different approach, developing a tightly focused core DSL with the software product itself being Envision and its capabilities. This allows them to create custom implementations for each client using Envision, while the compiler and environment for Envision are implemented in C#, F#, and TypeScript.
Vermorel explains that using a DSL can introduce limitations, but this can be advantageous in certain situations. For example, when optimizing a large supply chain with a sizable dataset, it can be difficult to ensure that calculations will be completed within a specific time frame using generic programming languages. A DSL with proper limitations in place can guarantee that calculations will be completed on time, preventing disruptions in the supply chain.
However, developing a DSL can be a time-consuming process. Vermorel shares that it took his company nearly a decade to develop their own DSL. This lengthy development time can be at odds with the fast-paced nature of startups. The key challenge in developing a DSL is rethinking the core problems to be solved and defining the logical primitives needed to solve them. This involves designing the programming language’s syntax and operators so that they are aligned with the problem at hand.
Despite the challenges, Vermorel believes that the DSL path has been underexploited and could be beneficial for startups. Developing a DSL does not replace the need for software engineers or mainstream programming languages; in fact, it may require even more software engineers. However, by focusing on a specific problem domain, a DSL can offer advantages in terms of efficiency, security, and optimization.
Vermorel begins by discussing the limitations of traditional enterprise software, which often becomes bloated and difficult to manage. He then introduces the concept of DSLs as a potential solution to this problem. DSLs are programming languages tailored to specific domains or industries, offering specialized capabilities and optimizations.
Vermorel points out that many software products in the market today are unsatisfying and tend to become like large enterprise software, which is not the ideal outcome. He provides the example of multi-channel order management systems (MOMS), which have grown to resemble enterprise resource planning (ERP) systems with hundreds of screens and thousands of options. The initial goal of differentiating from ERPs has been lost, and the resulting product is not much better than the original ERP.
He argues that using DSLs could have led to a leaner and more powerful product in the case of MOMS. However, adopting a DSL comes with its own challenges, such as years of headaches and constraints. On the other hand, using a more mainstream programming language might allow for more rapid growth but could lead to an unmanageable product.
One successful example of a DSL is SQL (Structured Query Language), a programming language used for managing relational databases. Vermorel notes that when a DSL becomes very successful, people often forget that it is a DSL in the first place. He believes that there is significant potential for DSLs in various industries, including supply chain optimization with Lokad.
Chandler inquires about other industries where DSLs could be beneficial. Vermorel suggests marketing as one possibility, where companies often struggle with complex software solutions that are not powerful enough to meet their needs. Human resources is another area where DSLs could offer a tailored solution, as it reflects the unique culture of each company, making it difficult for one-size-fits-all approaches to be effective.
Full Transcript
Kieran Chandler: Today we’re going to find out a little bit more about how they’re developed and also understand why they can be advantageous when compared to some of the more mainstream programming languages. So Joannes, perhaps you could start by just telling us a little bit more about what DSLs actually are and how they actually work.
Joannes Vermorel: DSLs are a specific class of programming languages that, unlike general programming languages like Java, Python, and C++, are not designed to be a solution for everything you can program on a computer. Mainstream programming languages are about giving you the most convenient, productive, and efficient way to do everything you can do with computers or many computers if you want. But domain-specific programming languages are something else altogether. It’s programming, so it comes with code, and it’s formal and abstract, but it’s designed to solve very specific types of problems. For example, historically, the first domain-specific programming languages were mostly about things like AMPL, a mathematical programming language, designed to do very specific tasks.
Kieran Chandler: So, what kind of problems would you be using a domain-specific language for, and what are they used for in the real world?
Joannes Vermorel: In the real world, one historical application would be to implement critical components. For example, you want to have a piece of software that is going to drive your ABS in your car, the anti-lock braking system, and you want to have proof that this thing is never going to crash because if it crashes, your car suddenly doesn’t have brake capability anymore. So, that’s a situation where it’s pretty dire, and you would think, let’s try to not have something too buggy here. That’s on the embedded side. Then you have problems like HTML for web pages, where it’s a programming language, but you want to have this thing more accessible. There is a good reason why you can learn HTML in primary school; it’s because it’s very simple. The basics are literally very accessible to even people who are fairly non-technical. But the trade-off is that HTML lets you control the layout of a web page; it does not let you control a robot or do artificial intelligence.
Kieran Chandler: So, they’re very simple and more constrained. I mean, why are you not using more mainstream programming languages, which have more expressivity for these tasks because they’re capable of doing it?
Joannes Vermorel: If you think about what it would mean for HTML web pages, for example, instead of just having a markup language where you can say “title,” “big font size,” and “body of text,” “I want to have the text justified,” and whatnot – simple controls – you would have to think, “Oh, I’m going to control every single pixel on my screen,” and that’s not practical.
Kieran Chandler: …would be what you could get from a super low-level approach, and if you want to be even more low level, you say, well, I’m going to directly control my graphic cards to have super high performance, and that’s maybe what, if you’re actually building a 3D engine for video games, you would end up doing. But if you just want to do something simple, such as a webpage, it would just take an infinite amount of time to do it that way. Through 3D video games, it’s so much simpler and much more direct to just go through IDs and such as HTML. Okay, and it’s a topic we’re very familiar with at Lokad, given we kind of generated our own DSL. So why was it something that was so interesting to us as a supply chain company?
Joannes Vermorel: The thing that is kind of maddening with supply chain is that problems are so diverse. It’s literally that there are no two companies alike, I mean, not exactly alike. They don’t have the same applicative landscape. Some companies have one ERP; many have two ERPs for bad reasons. They have a WMS; they have multiple WMS. They have 10 different ERPs because they operate in ten different countries with different IT landscapes. They have the e-commerce platform that came later, which is a separate thing. They have extra accelerators. So the problem was, we wanted to do supply chain optimization, and I realized during the early years that literally, the classical approach of having a framework that you could configure would just not fly. I mean, situations were just so diverse that we were ending up with a massive amount of customization and configuration for every single client. And literally, when you start thinking of having software that’s completely off the shelf but it just takes six months to do some configuration with the software, is it really configuration? Is it not like literally reinventing a new piece of software? The reality is, yes, you do. And so, we decided to take this angle to the next logical step, which is, okay, if every time we face a situation, it requires a piece of software that is different, what about having something that would support this process to make it very productive, very fast, very reliable? And actually, that’s where the idea of having a DSL, domain-specific language, came from.
Kieran Chandler: So, if you’ve used a more mainstream programming language, that configuration would take a lot more time, whereas having that kind of constrained environment means you can do things a lot quicker.
Joannes Vermorel: That’s interesting. I mean, at Lokad, we are using generic programming languages. We have been from the very start using C-sharp, C-sharp.NET, which is basically the programming stack of Microsoft, and later on, we added F-sharp and TypeScript for various purposes. So, we are already extensively using generic programming languages, and I’m very familiar with what you can do with those languages. But we realized during the first years of Lokad that tackling client requests with these sorts of programming languages was just incredibly sluggish and costly. So, we needed something that was better, and it was not the problem that we were not familiar with those programming languages. We realized that going through them for every single client situation was slightly nightmarish. And by the way, that’s also one reason why all those supply chain software solutions end up like gigantic monster pieces of software. It’s because they try to put into their software product everything, and then you end up with a software monster. So, we decided to say, okay, what if the software product was
Kieran Chandler: What if the software product was just a DSL and its capabilities, so something that would be like a super-tight core, and then when we go to a client, then we will create a custom implementation implemented not in C-sharp but in Envision, our own DSL? But the compiler and the environment for Envision, they are actually not implemented in Envision, they are implemented in C-sharp, F-sharp, and TypeScript.
Joannes Vermorel: Okay.
Kieran Chandler: You mentioned that obviously supply chains are incredibly diverse and every client is incredibly different. Does using a DSL introduce any kind of limitations? Does it stop you from actually implementing certain things?
Joannes Vermorel: Absolutely, and that’s the whole point, as surprising as it might be. You see, for example, one very basic problem that you face is when you have a large supply chain that you want to optimize and you end up with a fairly sizable dataset. Let’s say you have one terabyte of data. It’s not absolutely huge; you can go into a supermarket and buy a 1 terabyte hard drive for something like 100 euros, which is fairly cheap. So, it’s a large amount of data, but not gigantic. Now the problem is your dataset is going to be updated every day, and you want to, for example, do a pass over this dataset to take smart supply chain decisions, like restocking decisions and pricing decisions.
Now, the problem is that if you have a generic programming language, how do you ensure that the computation is going to only take, let’s say, less than 60 minutes? It’s very hard. As soon as you have arbitrary loops or constructs, it becomes super hard to prove that your execution is going to fit within a specific timeframe, which can be a problem if the calculation you’re doing to take certain decisions, such as restocking decisions, need to fit within a tight sequence of execution within your ERP systems. You really need to have this execution done in 60 minutes; otherwise, you disrupt your entire supply chain because the calculation takes too long.
So that’s typically the sort of thing where general programming languages cannot give you these guarantees because precisely you can do everything with them, so it’s very hard to get any type of guarantees from those programming languages. But with a DSL that has proper limitations in place, there is another angle at the moment: design. It’s actually very hard to offer a programming environment that is completely secure with a generic programming language. With generic programming languages like Java, Python, or C-sharp, you open yourself to entire classes of security vulnerabilities. If you can do anything with a computer, you can do things that are relatively dangerous from an IT security perspective.
Again, having a DSL means that there are entire classes of things that are just not even accessible to you, such as playing with OS, and thus it removes entire classes of problems that are not even your concern. All you care about is supply chain optimization.
Kieran Chandler: Yeah, and that’s what we kind of spoke about in our security episode. Let’s look at maybe the development of a DSL. I mean, how long does it actually take to develop one of these languages, in your position, how long did it take to develop Envision?
Joannes Vermorel: That’s a good question. It literally takes a decade, which is kind of insane. So when you’re a start-up, you know, you say move fast and break things, let’s have a minimum viable product that you can sell within six months, and then you start crafting your own DSL, and it’s like literally a process of
Kieran Chandler: Certainly, a multi-year effort and the key challenge is that you really have to rethink at the core the problems that you’re solving. What are the fundamental aspects of the problems, and what should be the logical primitives that need to be accessible to your mind to solve those problems with computers? It’s even worse than just inventing new words; it’s inventing logical primitives for connecting concepts so that you can, in the end, frame entire classes of solutions into this language to get solutions generated in ways that are fully automated by computers. But you’re literally thinking about the programming language itself, its syntax, the type of operators you have, which is… and you really want to have those things completely aligned with the problem that you’re trying to solve. So if you were a start-up starting out now, would you recommend setting out on that path and developing this language, which can be incredibly time-consuming, quite difficult to do, or would you more advise staying put and working with more mainstream programming?
Joannes Vermorel: First, crafting your DSL is not something that you will use as a substitute for mainstream programming languages. If you’re a software company and you want to create this new language as a way to solve a class of problems, as we do for supply chain, you will need a compiler and a runtime to execute these programs written in these programming languages. And this compiler will be written with a regular programming language. So it’s not because you go down the DSL path that you will not need software engineers; quite the contrary, you will need even more software engineers.
Considering the question for start-ups, I think it’s interesting because the DSL path is so ambitious that it has been significantly under-exploited. I see many software companies and start-ups trying to address problems the classical way because they are in a hurry, and they end up with slightly unsatisfying products. When I look at the type of products they are delivering to the market, I say it’s interesting, but they’re heading straight toward becoming a big piece of enterprise software, which is not exactly where you want to go.
One example would be what I call “multi-channel order management systems.” There is a wave of software products that went down that path, and the bigger ones now are starting to look pretty much like ERPs on their own, with literally hundreds of screens, thousands of options, and it takes months to properly set them up. They are not actually ending up in a place that is so much better than the ERPs that were the starting point for differentiating themselves to have products that were leaner, faster to deploy, and so on. Ten years down the road, you end up with something that is incredibly similar to an ERP, and maybe that’s typically the sort of problem where using a DSL would have made a difference.
Kieran Chandler: We’re discussing the differences between using a domain-specific language (DSL) and a mainstream programming language for software development. With a DSL, you might face years of headaches but eventually end up with a powerful and lean solution. On the other hand, using a mainstream programming language could lead to more rapid growth but could result in an unmanageable system.
Joannes Vermorel: It’s interesting to note that one of the first successful DSLs was SQL, the query language for databases. Nowadays, every database vendor is essentially selling a DSL, as the only way to interact with a database is through queries written in a domain-specific language. When a DSL becomes incredibly successful, people forget that it’s even a DSL. For example, HTML has become so prevalent that people don’t think of it as a DSL. I believe there’s a lot of potential for DSLs in various industries, like supply chain optimization with Lokad.
Kieran Chandler: Apart from the supply chain industry, which other industries do you think could benefit from using a DSL?
Joannes Vermorel: Marketing is one industry that comes to mind. I see many companies struggling with complex software solutions that aren’t powerful enough. They end up doing a lot of work with Excel, which is difficult to maintain and pass into production. Human resources is another area where DSLs could be beneficial. Human resources management often reflects a company’s culture, making it hard to have a one-size-fits-all solution. I believe DSLs have the potential to make a significant impact in virtually every industry, but the way they are implemented can vary greatly from one problem to the next.
Kieran Chandler: We’re going to leave it at that. Thanks for your time today, Joannes.
Joannes Vermorel: You’re welcome.
Kieran Chandler: That’s everything for today. Thanks for tuning in, and we’ll see you again next time. Bye for now.
