Kirill Eremenko: 00:00:00
This is episode number 419 with master software architect Juval Löwy.
Kirill Eremenko: 00:00:12
Welcome to the SuperDataScience Podcast. My name is Kirill Eremenko, Data Science Coach and Lifestyle Entrepreneur. And each week we bring you inspiring people and ideas to help you build your successful career in data science. Thanks for being here today, and now let’s make the complex, simple.
Kirill Eremenko: 00:00:44
Welcome back to the SuperDataScience Podcast, everybody. What an episode I was blown away, my mind destroyed with this information that Juval shared this now. So Juval is a master software architect. He’s been in the industry for decades and he’s developed a unique method of writing software in a way to avoid it becoming outdated. In a way for avoiding for it to start not working, avoiding any incurring massive technical debt, a term that’s become common place in the industry now. And in this podcast, he shared over 10 years of his experience and thinking about how to write software and what is taught to other people. And he runs masterclasses that are sold out way in advance, and that are really hard to get into.
Kirill Eremenko: 00:01:52
He shared all that in an hour. He shared some really cool insights from his new book, Righting Software, which is spelled R-I-G-H-T-I-N-G. This podcast even though it’s about software design, it is incredibly useful to data scientists. I found a lot of value in it, especially if you’re not only doing insights, one off insights, but you want to then integrate, put your models and your things that you create in data science, You want to put them into production. This is going to be extremely valuable if you are in any way linked to product design, or you want to add product design to your toolkit, which is a very powerful thing when you combine with data science, this is the podcast for you.
Kirill Eremenko: 00:02:42
So here are some of the things that we discussed today. First you’ll hear about how Juval planned his career years in advance before it all happened. It was a very interesting story. Then, why designing against requirements is a very sure fire way to kill your project. You’ll learn about fractal systems, technical depth, good designs, and why they have to have volatility. You’ll learn about reusable interfaces, commonalities between systems. So, that was all about a software system design. Then we talked about project designer. As if that was not enough, Juval will surprise you with another whole, probably like 15 minutes of super valuable information. You’ll understand why project design is even more important than good system design.
Kirill Eremenko: 00:03:37
We’ll talk about time, cost and risk, and how those three play together in the equation of project design. And then he’ll finally put altogether in the discussion about bundling system design and project design together. This podcast for me was a huge, eye-opener to the world of software system design and project design. And I really believe this is a unique opportunity to learn from one of the top people in this industry. And moreover Juval’s mission is actually to fix the entire software industry there is a lot of things that have been done wrong. So if you’re already in the software industry, you will find lots of valuable insights here. If you’re in the data science industry, but haven’t worked in putting projects into production or in the software components of data science, then this podcast will allow you to start off in the right foot, into this space. Super pumped for you to check out this episode. So without further ado, I bring to you master software architect, and I forgot to mention and founder of IDesign, which is at idesign.net Juval Löwy.
Kirill Eremenko: 00:04:55
Welcome back to the SuperDataScience Podcast everybody. Today, we’ve got a special guest calling in from Los Gatos, which not many people know where it is, but it’s in the Bay area in California, Juval Löwy. Juval welcome.
Juval Löwy: 00:05:08
Thank you. Pleasure to be here.
Kirill Eremenko: 00:05:11
What brought you to Los Gatos?
Juval Löwy: 00:05:14
So a few decades ago in the late 90s I was the chief software architect of a Fortune 100 company based in Silicon Valley. And it’s not an accident because I actually planned my career. And one of the things I had to do was be the chief architect of a Fortune 100 company. And here you can sell a stone and you hit a few companies it’s not even special. And I stayed here, it’s the Bay area, the weather, the people, I mean, what can you do? It doesn’t get much better than that. Maybe you live in Australia and you can actually improve on that. But it’s one of the nicest places in the world.
Kirill Eremenko: 00:05:55
How big is Los Gatos? How many people?
Juval Löwy: 00:05:58
It’s small in American scale. So it’s like 30,000,
Kirill Eremenko: 00:06:00
30,000?
Juval Löwy: 00:06:02
Yeah.
Kirill Eremenko: 00:06:02
Wow. That’s actually quite big. I’m surprised not many people know about it.
Juval Löwy: 00:06:07
It’s stacked against the heels. So at the end of my street, literally the hill start and we actually have got two places here. Also, I have a farm. A 100 acre farm where I do a lot of work.
Kirill Eremenko: 00:06:22
I would like to have a farm one day. It sounds like a nice, relaxing thing to do.
Juval Löwy: 00:06:27
It’s quite demanding and backbreaking, but it’s very rewarding.
Kirill Eremenko: 00:06:31
Rewarding. It helps you get away from all the social media and just focus on physical work.
Juval Löwy: 00:06:38
It’s more than physical work. It’s planning and thinking ahead and all the things that as an engineer I like to do. So I’ve a very engineered farm.
Kirill Eremenko: 00:06:50
Okay. Okay. Got you. Farmer of the future. In relation to San Francisco, whereabouts is Los Gatos?
Juval Löwy: 00:06:56
It’s about an hour South of San Francisco.
Kirill Eremenko: 00:06:59
Okay. I knew another big city, a bigger city there.
Juval Löwy: 00:07:05
San Jose.
Kirill Eremenko: 00:07:07
San Jose.
Juval Löwy: 00:07:07
Yeah. So outside the Bay I just say I live in San Jose because everybody knows where San Jose is.
Kirill Eremenko: 00:07:12
Okay. Got you. Okay. Awesome. Well-
Juval Löwy: 00:07:15
And for the foreigners, it’s the other airport in the Bay area.
Kirill Eremenko: 00:07:23
Exactly.
Juval Löwy: 00:07:24
But there’s an interesting point here, which we may want to discuss, which is career planning. I mean, people design systems and such, but nobody stops to take the time to design themselves, which is a really important point, right?
Kirill Eremenko: 00:07:41
When did you, or how did you come up with the idea to be a chief software architect at a fortune 100 company?
Juval Löwy: 00:07:48
So I wanted to actually do IDesign. I wanted to found IDesign. Okay. So that was the end goal, but I had to-
Kirill Eremenko: 00:07:56
Take us back. How many years ago was this?
Juval Löwy: 00:07:57
That was I would say 28 years ago.
Kirill Eremenko: 00:08:02
28? And when did you found IDesign?
Juval Löwy: 00:08:05
In 2000.
Kirill Eremenko: 00:08:07
In 2000? So eight years before you founded the company, you wanted to found, you already had a plan?
Juval Löwy: 00:08:13
Yes.
Kirill Eremenko: 00:08:14
Okay. And how does being a head architect in a fortune 100 company fit into that plan?
Juval Löwy: 00:08:24
So you both need the credentials, you need the street creds, and you also need the experience, right? Because if that’s the kind of work you want to do, then you can’t say, “Well, I’ve never done this kind of work, but I really want to.” We live in a world where it doesn’t matter what it is, being a doctor, a lawyer, a pilot, an architect, you only get to do the job if you already do the job, if you have experience doing the job. You wouldn’t want to be the first patient of a doctor. And you wouldn’t be the first passenger on a new pilot. So there’s an ongoing combination. And the stepping stones even before that. And the journey actually started I would say around 1990, when I observed something which was so fundamental, it took me almost a decade to actually understand what I actually observed.
Kirill Eremenko: 00:09:11
What was is it?
Juval Löwy: 00:09:14
That you never design against the requirements.
Kirill Eremenko: 00:09:18
And what does that mean?
Juval Löwy: 00:09:19
Okay. So if you look at how most people design software systems, they try and maximize their adherence to the requirements. So they say the system should do A and B and C and they put sticky notes of A and B and C on the Kalman board. And then they code A, they do absolutely what they can to manifest A in code. And then they do a B, and then they do a C. And I realized that if you do that, that is this and that is the reason why most software systems out there are complete failures.
Kirill Eremenko: 00:09:51
Okay. Are you subscribed to the Data Science Insider? Personally, I love the Data Science Insider. It is something that we created. So I’m biased, but I do get a lot of value out of it. Data Science Insider of if you don’t know, is a absolutely free newsletter which we send out into your inbox every Friday, very easy to subscribe to. Go to www.superdatascience.com/dsi. And what do we put together there? Well, our team goes through the most important updates over the past week, or maybe several weeks and binds the news related to data science and artificial intelligence. You can get swamped with all the news, even if you filter it down to just AI and data science. And that’s why our team does this work for you. Our team goes through all this news and finds the top five, simply five articles that you will find interesting for your personal and professional growth.
Kirill Eremenko: 00:10:49
They are then summarized, put into one email and at a click of a button, you can access them, look through the summaries. You don’t even have to go and read the whole article. You can just read the summary and be up to speed with what’s going on in the world. And if you’re interested in what exactly is happening in detail, then you can click the link and read the original article itself. I do that almost every week myself. I go through the articles and sometimes I find something interesting, I dig into it. So if you’d like to get the updates of the week in your inbox, subscribe to the Data Science Insider, absolutely free at www.superdatascience.com/dsi. That’s www.superdatascience.com/dsi. And now let’s get back to this amazing episode.
Kirill Eremenko: 00:11:31
Why is that death?
Juval Löwy: 00:11:33
It’s death because requirements change.
Kirill Eremenko: 00:11:36
That what, the climate change?
Juval Löwy: 00:11:39
Requirements change. Now, requirements change on its own is a very good thing. And the reason it’s a very good thing is because it keeps all of us employed. You, me, the listeners, if requirements were static, none of would have a job because somebody, somewhere would write a software system that does what they want. And that will be it. But because of requirement change, we are here. And in fact, it’s a very good thing because there’s so few of us and so many of them. And as the more requirements would change the higher the demand for our services, right?
Kirill Eremenko: 00:12:12
Yeah. True.
Juval Löwy: 00:12:13
And higher demand is high compensation, which is also good. Right?
Kirill Eremenko: 00:12:15
Yeah.
Juval Löwy: 00:12:16
So requirement change is a very good thing. Now, if most software systems are built against the requirements, and that’s another way of saying they’re designed against the requirements. They actually reflect the requirements and the design of the system.
Kirill Eremenko: 00:12:32
Oh, so you mean against not opposite, but against as-
Juval Löwy: 00:12:37
Reflecting, like a mirror. You’re standing in front of a mirror and you reflect the requirements in the architecture. So if the requirements are A and B and C, you will have an A block, a B block, a C block or you’d have a block that does A and B and C. It’s the same thing. Right? And so whenever you reflect the requirements in the design, when the requirements change, the design has to change. Right? That’s obvious. It turns out that in software and in life in general, once you’ve already put the effort into building something, and then you need to change the design, it’s the most painful thing you can do. If you haven’t built anything, changing design is almost free, but if you already built something it’s very expensive. For example, suppose I were to figure out if it’s possible to build a house right by strictly doing it against the requirements?
Juval Löwy: 00:13:26
Well, there’s requirements on how to change the house. And that’s totally insane. Now, once I realized that, I realized okay, so if that’s the pattern, then what can you do about it? And I observed that you should never design against the requirements. And I started unraveling this swaddle. I started pulling on the swaddle and say, “Okay, so what does it actually mean?” And I started looking at other systems and not just office systems. And I observed that in the world around us, nothing is ever designed against the requirements. Now, it also doesn’t make business sense I have to tell you. And the reason is if you only design against the requirements, you cannot innovate. Henry Ford said that if he listened to his customers requirements, all they would want is a faster horse.
Kirill Eremenko: 00:14:12
Or like a metal horse, right?
Juval Löwy: 00:14:14
Or a metal horse. I want a horse that doesn’t need feed or something. And initially people called cars even horseless carriage because they couldn’t wrap their mind on anything besides a carriage. So first of all, you cannot innovate if you listen to your customers or to their requirements. And you know, nobody before to 2006 came to Steve Jobs and say, “I want an iPhone with an app store.” I guarantee you that no customer ever said that sentence. So if you only designed against the climate, you cannot innovate. So, that’s one problem. Okay? But that doesn’t mean you shouldn’t listen to customer. You should absolutely get customer feedback on refining what you’re doing. So if this is what I want to do, would this be a good way of doing it? That’s a great way of getting customer feedback. But you never ask the customer, what should they do? Because, they would want a faster horse.
Juval Löwy: 00:15:07
So that’s a problem. That’s just the beginning of the problem. If you look all around us, doesn’t matter if we’re talking about cars or laptops or anything, or airplanes or houses, nothing is ever designed against the requirements. For example, a car is a very simple requirement. Take you from A to B, you agree?
Kirill Eremenko: 00:15:24
Yes.
Juval Löwy: 00:15:24
You wouldn’t buy a car that doesn’t support that requirement.
Kirill Eremenko: 00:15:27
No. It’s not a car.
Juval Löwy: 00:15:28
But there isn’t a single box in the design of the car that takes you from A to B. Nothing. If you think about it, nothing in the car takes you from A to B. Now the car has gearbox and engine block and radiator if it’s gasoline and a battery if it’s electric. Still, none of these things take you from A to B. It’s the integration of those things that take you from A to B. So in fact, nothing in the design of the car is about taking from A to B. Right? All of the design in the car is about doing other things like storing electricity or internal combustion. Look for example at the human body, right? So the design of the human body emerged on the plains of Africa 200,000 years ago. We haven’t changed in 200,000 years. Genetically identical to Pre-Neolithic hunter-gatherers.
Juval Löwy: 00:16:20
And I’m pretty sure that being a software architect wasn’t part of the spec at the time. So the question is, how could I possibly have the same architecture using a Pre-Neolithic hunter-gatherer? And the requirements have drastically changed, but the design did not. If you look at the saber-toothed tiger, the tiger with the big teeth. That one was definitely designed against the requirements. The big teeth was to cut the throat of the American Mastodon, which is a kind of a small elephant. Now, once the early humans came and hunted down all the elephants, the saber-toothed tiger became extinct because it was designed against the requirement. It wasn’t to start raising wheat and chewing leafs, or do anything besides killing mastodons. The requirements, change system death, if you design against the requirements. And I can give you lots of these examples of how nothing good meaning you can have bad designs, you never designed against the requirements.
Juval Löwy: 00:17:25
And I observed that in 1990, not even in the software context. I observed it in a different context.
Kirill Eremenko: 00:17:31
What context?
Juval Löwy: 00:17:33
You can call it the context of designing very large systems. But I cannot disclose the specifics, so we’ll just leave it at that. And the organization in question always had to come up with design solutions under extreme time pressure for very complex solutions that kept changing. And the time it took to design against the requirements was actually longer than time it took the requirements to change. Which is a non problem in software today. Because by the time you pick the sticky note of the Kalman board and code it, it has already changed, right. Or maybe you got it wrong. And if you think about it, it’s not even just the change. Nobody in the history of the world has ever had the time to beautifully spec all the requirements before you started the work ever.
Juval Löwy: 00:18:27
So whatever requirements you have is by definition flawed. Because, nobody had the time. And even if they had the time, they typically have duplicates. We need to do this, and we need to do that, and the reason is, if requirements come from customers, then when the marketing person talked to one customer they describes something and they put it in the requirements and then talk to another customer, they describe the same thing, but in a different vernacular. So they put that in the document too. So if you design against the requirements, you do double the work, which is insane. And then you have things in the requirements which are mutually exclusive. So one customer wants it black, the other one wants it white. I can do black or white, but I cannot do both. So what do you do now? And then regulations change.
Juval Löwy: 00:19:09
And then they have requirements that you will require to do. But now the government says, “You are not allowed to do this ever again.” Or “The new privacy, whatever you cannot store it like this,” or whatever. So now you’re not even allowed. You have to actually throw away certain requirements. So whatever you are given in requirements is always garbage. It’s incomplete. It’s inconclusive. It’s half baked. It’s full of mutually exclusive and contradiction. It has duplicates. It’s garbage. Now, you don’t have to actually be a genius to realize that if you have a machine, we can call the machine your project. And you feed garbage on one side, garbage comes to the other side, right? So if I give you requirements and it’s garbage and you do some processing on it and you call it your system and something comes out of the other side, all you got is garbage.
Juval Löwy: 00:19:57
And the notion of garbage in garbage out is not new to data scientists I hope. Right?
Kirill Eremenko: 00:20:02
Yeah, true.
Juval Löwy: 00:20:04
Your data model is only as good as if your data is not garbage. It’s as simple as that. And how much time people spend cleaning up the data, why does they clean up the data? Because they got dirty data, which is garbage, right? So in the abstract if you build your system and design it against the requirements, you will get garbage on the other side. Now people always know it.
Kirill Eremenko: 00:20:24
Sorry, just to clarify. So it’s not just about garbage data in garbage results out, it’s also the garbage structure of the system that you’re building?
Juval Löwy: 00:20:36
That’s the garbage that comes out, right?
Kirill Eremenko: 00:20:38
Yeah. So you can have great data coming in, but if your requirements were bad and then you have a garbage system that processes the data, you’ll still have garbage?
Juval Löwy: 00:20:47
Look, the audience as data scientists is people that deal with data and such the data itself is valueless. You don’t do it, I mean, maybe in academia, you can write a nice paper about doing something because it’s interesting. And it’s got a nice algorithm and such, and it might not be valuable or useful, and that’s fine in academia. But in the commercial world, you don’t do big data or data analysis and such for its own sake. You do it because some customer needs these to do some business objective, that’s it. That’s why you do it. If nobody wants it, you’re not actually going to do it. That is the proof. Saying, if nobody wants it, I’m always tying it back to the requirements. So even in a data centric project, you are feeding off the requirements. So I can draw another box, even bigger around the data and say, okay, so this box get some garbage in and guess what comes out?
Juval Löwy: 00:21:41
In the notion of garbage in garbage out, how big is the box, doesn’t matter. It’s the fact that you inputted garbage, that’s the problem. And I observed that this is true in all system, biological, artificial like cars. And I also started observing also how do things actually work? If you think about it, how do things actually work? And I observed that it’s actually a fractal observation. And so if you look, say at the laptop I’m using now to provide these session. So there is actually a session going on right now. And the session requires the integration of Kirill, Juval, the internet, my laptop, your microphone, and call done in the cloud and everything else. You take out one item out of it, and the whole thing is dead.
Juval Löwy: 00:22:34
You cannot take one of these essential items. Now you can add things which are needless, but you can’t take the essential things. You agree?
Kirill Eremenko: 00:22:43
Yes.
Juval Löwy: 00:22:43
Okay. So, and yet none of these things actually provide the business value of the presentation we’re doing now, the discussion. The presentation is the aspect of integrating all of these things together. Now, when I’m saying the assignment is fractal look at my laptop. My laptop actually doesn’t know I’m talking to you right now. It really doesn’t know we’re on Zoom. It doesn’t know any of these things. The laptop just needs to send and lots of zeros and one out of the network card, that’s it, that’s all it needs to do. So as far as the laptop is concerned, the feature is sending some network communication. It’s not doing a webcast. Now, if I look at the laptop with the system, it has one feature which is sending the network communication, but there’s no one thing in the laptop that does the communication.
Juval Löwy: 00:23:32
There’s no one box it components say, “Aha, that’s where the network is.” Why? Because I have to integrate the CPU, the memory, the bus, the hard drive, the network card, even the box of the laptop has to be integrated for me to do that. There isn’t a single thing and left with what does the network. Okay. So I mentioned the hard drive. So the hard drive is storing some files, which are actually required like the drivers for the microphone and everything else. Now the hard drive doesn’t know about network communication, as far as a hard drive is concerned, there’s just one feature, which is storing stuff. That’s it. That’s the only thing the hard drive knows about. But if I look at the hard drive system, there isn’t a single thing in the hard-drive doing the storing, right? The hard drive has some media for storage could be completely solid state, could be some spinning things.
Juval Löwy: 00:24:22
It has a voltage regulator. It has its own bus by the way. An internal bus inside the hard drive, it has its own clock. It’s got its own controllers and then all of it is very cleverly put inside the little box with little screws holding everything that you nead like this tiny screwdriver to take them off. And the integration of all of these things, even the screws provide the feature of storage. Now, if I look at the screw, the screw is one feature. If I look at the screw as a system, it has one feature, which is, you know what it is? It’s fastening. It needs to fasten things. It doesn’t know about storage. But there is no one thing in the screw that’s doing the storage because I have to integrate the sled on the screw and the stem of the screw, where the tension is actually held and the head of the screw and some tok to provide the feature of fastening.
Juval Löwy: 00:25:09
Now you can drill this way, all the way down to the quarks, and you will never see a requirement. You will never see a feature. Now, that’s how the world is put together. Doesn’t matter if it’s a laptop or my body. Now in the crazy world of software, people pretend that they can actually code a feature. That they can actually just take a behavior and put it in code and that’s the system. But we’ve just discussed, it goes against the nature of the universe because in the real world, that’s not how things actually operate. Now the result is, yeah maybe the first version works sort of, because even then it works against requirements which we know are flawed. I’m saying sort of, certainly by the time requirements change enough, it is completely unworkable.
Juval Löwy: 00:25:54
And then people stopped patching up the code and connecting this to that and working things. And they create such a big mess. At some point they throw their hand in the air, the say, “We can’t take it anymore. We have to do a new system.” Why? I’m using a 200,000 year old system and I’m using it very efficiently. And then they talk about technical debt as if the gods have ordained that software will have technical debt. But nobody bothered to ask why do you have technical debt? Why did it happen? What’s the root cause? But I just gave you the root cause. And it’s very simple root cause. It’s like an apple falling from the tree right. And sometimes it does take the simple observation after lots of people in thousands of years have it wrong.
Juval Löwy: 00:26:39
You know what I mean? And so I had that spark in 1990 that you never designed against the requirements. And that you provide behavior by integrating components. Each one of them is doing something that has nothing to do with the supposed business value, ever. Okay? And then I said, “Okay, so now that that’s a design principle, what does that actually means? Right? What is the nature of those components? And it turns out that you can even find a master observation, a master abstraction of what all well designed components actually do. And it turns out that all well designed components encapsulate some form of volatility, some area of change. And then you start in the abstract view your design is a series of vaults. Each vault encapsulates some potential area of change. Now, when a change happens, it changes the requirements say, or it changes the business.
Juval Löwy: 00:27:43
Then the change is very risky. It’s like tossing a handgun in into your system. So you take the pin out and saw a hand gun into the system. Now think about it. If all your fuel components are areas of change of different types. Because you can talk about different types of changes as well. Then you open up the door of the appropriate volt, toss the handgun inside and close the vault and the vault does boof. Now, whatever was inside the volt may be completely destroyed but then nobody cares about anything else. And you have contained the change. Now, think about designing against requirements when you reflect requirements in your design. Because your association of building blocks or one big block and so on is all about requirements by-
Kirill Eremenko: 00:28:31
The ABC that you started from.
Juval Löwy: 00:28:34
The ABC by definition changes on never in one place, they’re spread split all over the system. So differently, when you design against the requirements, you maximize the impact of changes. But it doesn’t mean it’s impossible. It just means that you’ve maximized the cost. And have to spend the most amount of time, the most amount of energy, the most amount of pain to incorporate the change. While if you encapsulate volatilities and changes, you minimize it, doesn’t mean it’s straight. It just means you have minimized it. And everything’s put together this way. For example, I like to drink tea. I have tea here. If I were to go over the off-camera and pour it on my laptop right now, which I’m not going to do, this is purely a make believe exercise. Okay? Then the laptop is completely destroyed. Yes? You agree?
Kirill Eremenko: 00:29:22
Yes.
Juval Löwy: 00:29:22
But you know what? Let me reach outside the camera. I have a backup laptop and in two minutes from now, we’ll be continuing our way. We won’t mind the fact that I just still do it. Now, this is a very fine laptop. Okay. I spent several thousands of dollars on the laptop and given how cheap laptops are today, it’s hard to do it. Okay? So I went to the novel and I took the best laptop and I maximized all the options. Okay? So I took a laptop and it will set me up a pretty penny, but nothing else in my life will change. Because I’ve contained the change. You understand? It doesn’t mean it’s free. I still be upset spilling-
Kirill Eremenko: 00:30:03
It’s like you’re on a submarine and you have sections of the submarine. And if one of them has a hole and the water’s pumping in, they close the doors and then the rest [crosstalk 00:30:11].
Juval Löwy: 00:30:11
That’s not just with submarines. That’s also with boats, it’s called the bulkhead. So those weird doors inside ships, it’s in case you hit a iceberg, or get to piddle. You close the bulkheads on both sides, whoever is inside well, that stuff, but if you’re outside then you’re fine, okay? So everything is designed this way.
Kirill Eremenko: 00:30:30
In the natural world, you mean.
Juval Löwy: 00:30:32
I just showed you a laptop and it’s designed this way too. Right? Then I can also replace the memory even inside the laptop or replace the hard drive and all well designed systems encapsulate these volatilities. And again, it’s a fractal observation. I can replace components inside the laptop. I can replace the whole laptop. And next week you’re going to have a different speaker besides me, you replaced me. Right? So we encapsulate these things which change. It’s a fundamental observation about the nature of good designs. Now, designs which did not encapsulate changes became extinct.
Juval Löwy: 00:31:07
And by the way, 95% of all life forms since the dawn of time are extinct. There’s a reason why. They didn’t handle change too well. And things change and then they become extinct. There gets too cool and it gets too warm the meteor hits, whatever it is. Well, that’s unfortunate. Okay. And so what I’m describing to you and I’m just describing to you is if it’s as mature source, it took me I would say eight to 10 years to think about these things and kind of like mature it. And then you can take it a step further. So what does it mean to encapsulate change? So, first of all, you have to start thinking in terms of interfaces that post two components, right? And you start looking at things and you realize that if you’re trying to reuse components across core systems, then the only reasonable things that actually interface, it’s not the thing itself.
Juval Löwy: 00:31:57
For example, if I look at the chair I’m sitting on, it’s very different than the chair you’re sitting on. You agree?
Kirill Eremenko: 00:32:03
Mm-hmm (affirmative).
Juval Löwy: 00:32:03
In fact, we couldn’t interchange one bolt or one nut from my chair to your chair, but I can sit on your chair and you can sit on my chair, even on that completely utterly different. And the reason is the interface between the seat and my butt is identical to the interface between your seat and your butt. So we can actually reuse the chairs, okay? Look at the car. You can drive my car and I can drive your car. Now, not a single bolt between my car and your car is interchangeable. Certainly not things like gearboxes and computers, none of it is interchangeable. And why could I drive your car? It’s because all the volatility behind the cars is encapsulate behind standard interfaces, like the steering wheel and the accelerator and the brake. So if we focus on reusable interfaces, we can change the underlying thing. So the thing itself is not reusable, the interface to it is reusable. So, that’s the next observation. And then I realize that in all types of systems, you can talk about families of volatilities. For example, all internal combustion cars, have a water pump.
Juval Löwy: 00:33:13
It wouldn’t work. There’s a VW Beetle, but that’s an extreme example of kind of like proves the case that most cars have a water pump, certainly all cars have a fuel pump. They don’t use gravity to pull the fuel from the tank because they have to have the fuel tank very high. And if one goes from filter then, so if you observe that, you can say, “Okay, so if I’m looking at houses, all houses have this. If I’m looking at laptops or laptops have that. If I’m looking at airplanes, all airplanes have that. If I’m looking at biological entities, I see the same thing.” For example, a mouse and an elephant have the same design. You agree? Exactly the same architecture, a mouse and elephant. Now the details are very different, but the architecture is the same. There has to be something very fundamental in the architecture that made it.
Juval Löwy: 00:34:01
So otherwise we would see different architecture between mouse and an elephant. Now a mouse and a grasshopper have different architectures, but I can spun it all the way from a mouse to an elephant and not change the architecture. There’s something very fundamental in this architecture. So my point is that regardless of systems and regardless of even types of systems, you can actually identify commonalities. Now, what I realized is that that commonality is families of volatilities. Families of things that could actually change. So again, to use the body, there’s enormous volatility in pumping blood. You can have high blood pressure, low blood pressure, high pulse, low pulse, dehydrated, not dehydrated. With one general means, other general means. Sleeping, resting, walking, running, being fired at whatever is, enormous volatility. Yes. But all that is encapsulated behind the heart, right? Okay. I can make some arguments about metabolism and processing oxygen and everything else.
Juval Löwy: 00:35:00
So going back to software system, you can actually identify families of volatilities in software system. If we’re looking at any system, and I’m not talking about digital programs, even though you can make some arguments for simple problems. But certainly by the time we talk about systems. Something that people are willing to pay money for. Not toys. Then very common sets of volatilities emerge. For example, all systems need to access resources, right? Because you have to store stuff, you have to queue up stuff, right? I mean, you won’t get away with not having some resources. And so, one fundamental volatility is what is the resource? It can be a database in the cloud, can be hash table, can be a file, right? So there’s volatility in what is the resource. Now, different resources have different ways of accessing it. You don’t access a database system where you access a file.
Juval Löwy: 00:35:56
Now, even if it’s a database, how many ways do you have of accessing a database? And the answer is 107. Okay. It’s like who knows? And Kirill what’s nasty about it is that none of those millions of ways is superior to all the other ways in every absolute respect. Because I can come up with something that only this one is doing better than everybody else. Right? So even if the resource is constant, the way you access it as volatile, right? So you know what? I need to encapsulate that. Now there’s, you can make a whole discussion that even the way you access the resource has to do with the fundamental behavior of the system, because you don’t access the resource for accessing sake, you’re trying to perform some operation on it. So there has to be some sort of a very low level atomic operation that your system needs to do.
Juval Löwy: 00:36:44
So if you look at your resource access there is a transformation function between those atomic business verbs and the actual act of accessing the resource. So now, if you change the way you access the resource or even change the resource. If you only explore that uniform interface, which is the atomic business verbs, nobody upstairs detonates, they are just fine with it. Now, you have to change the resource access, and that may cost you, but nothing else changes. Move it to the next level up. You have to do certain activities. Now, we can look at business activities and each activity may have different ways of doing it. So even activities themselves have volatility. Now, another level on top of that, we can say well, this system is not just doing an activity. The system is always sequencing a set of activities. If you recall back, we said that you have to do some requirements.
Juval Löwy: 00:37:34
Requirements at the end of the day are the service of business. Businesses needs to perform some set of required behaviors. In fact the requirement is not even to do a particular functionality like A. The requirement is always, here’s how I want you to do A. But it’s a required behavior. Now, a required behavior is always sequencing of activities. It’s not just a one thing. So if you look at sequencing of activities, I can do A, and then B and then C. But I could also do ABC in parlor, or even I do A, and then conditionally I do B and C. So while I’m still doing the same three activities, there is no more volatility in the sequencing of the activities, which is independent from the activities, volatility themselves. Right? So we have different access of volatility. I can change the sequence but not change the activities. I can keep the sequence change the activities, right?
Juval Löwy: 00:38:29
And the activities go and access the resources. So now on top of that, we can have some kind of an access interface layer, and at that level we can show information to users and using some kind of gooey or we could have some kind of an API, right? And a good API should also actually encapsulate the volatility of the underlying systems. Obviously, if I’m changing the way I’m sequencing the activities, nothing in the user interface would really change. I mean, that’s a really bad days that actually happens. Right? Because all the users are upset and they have to really pull the clients and so on. So it’s another way of saying that the top layer that exposes your system to the world and presents it to the world that means encapsulate the whole volatile of what happens underneath. And then there’s all the things which have nothing to do with your business.
Juval Löwy: 00:39:13
Why? Because every system needs security. Now, while every system needs security, security is not that different between my system and your system. If we decide that we are going to use username and password, you know what, there’s probably one good way of doing it and that’s it. And if we decide we’re doing single sign-on, there’s probably one or two ways of doing single sign on and that’s it. But there’s lots of ways of doing different ways of security. So security is volatile. And if we’re talking about connecting different subsystems, well maybe you want to connect it over a message bus, but you know what? Message bus is a message bus. I can take a message bus from my system, move it to your system. It may not be exactly the same, but it should be fine. Now different message buses use an API. So maybe I want to encapsulate that. Maybe I need to do some diagnostic.
Juval Löwy: 00:39:59
Maybe I need to do some regression testing, automated build, logging, instrumentation, analytics. Right? All of these things have nothing to do with the requirements, nothing to do with the business. And yet they are the onset of volatilities, which are kind of like a utility like. And the same is true by the way in the house, your house has utilities, it gets power and water and sewer. You wouldn’t live in the house without power. Now, power in the house is also very volatile because you can have AC and DC, 110, 220, solar panels, generators, connectivity from the grid and maybe a blackout. So all of these things are very volatile, but you don’t actually care because you encapsulate behind the receptacle on the wall. Crazy volatility on the other side of the receptical you don’t care. So give me power.
Juval Löwy: 00:40:49
So there’s a whole other family of volatilities and system, which use other utilities. And so you can absolutely identify families of volatilities from the resources to the sequence, to the utilities. And then you can take it another level and you can say, ‘okay, if there’s common types of volatilities, are there also common types of interactions?” Things that you should do, things that you shouldn’t do. For example, how about your database calling your user interface? Okay. That’s really a bad idea. Let’s not do that. How about your clients calling the database? Not as bad, but still very bad, because it just means that all the business logic now resides in the client. Right?
Juval Löwy: 00:41:27
So I can come up with interaction rules between these things. And the more structure you put on it this way, the more easy it becomes to do design because you don’t start with a blank slate, you have all this mental machinery of all the things the system should do. And you spend the time just customizing that for this particular context. And as a result, you can produce world-class designs that can withstand anything the customer or the market or the business, or the regulator can sort it for decades to come and you can do it very quickly. And I realized all of that by about 2000. And at that point I said, “Okay.”
Kirill Eremenko: 00:42:10
So you just compressed 10 years of realizing into 40 minutes. Wow.
Juval Löwy: 00:42:17
Yeah. And I literally spent close to a decade just thinking about these problems and trying different angles and different tasks because nobody ever gives you anything like this on a silver platter and say, “Use this.” I mean, I do it now in my recent book, but it was a journey for me. And I’m not just throwing numbers. I have boot into production systems that were good decades later because I’ve used these principles. And people will come to me and send me emails or meet me in person and say, “We’re still using your design from 20 years ago.” They changed technologies and this and that, barstool and elephant, the architecture stayed. And then I spent the next 20 years roaming the globe, not just myself, the other architects at IDesign, helping hundreds of companies use these techniques and ideas.
Juval Löwy: 00:43:07
And I taught these techniques and ideas to thousands of architects. So I do masterclasses all over the world and people literally come when I’m doing it in California, come from all over the world. And I am teaching these ideas and these techniques and every one of these people, when they go back they’re transformed because it’s like the blinders are off. They know what it takes to design software systems. And I didn’t stop 20 years ago because all of those hundreds of interactions with customers and projects kept giving me additional insight, additional observation, additional insight. And we started observing numerical patterns and ratios. I mean, there’s is a whole beautiful thing that came out of it. So it was like these, by the way, still just on the system side because I can make exactly the same argument on the project.
Juval Löwy: 00:44:00
And the project is this thing that everybody has to do, which is the combination of people and activities and time and risk and money. All of that needs to be put together. Now a good project is not anybody picking up sticky notes and start coding it. That’s this. Okay. It’s definitely because of what we discussed on the system design, but it’s just because of the project. And the reason is whenever you have multiple things to do and multiple people to do it with, there’s near infinite combination of doing it. Unfortunately those combinations are not equal.
Juval Löwy: 00:44:33
For example, one crazy combination is doing everything with one person sequentially, just the giant string. That’s probably the riskiest and the longest way of doing it, or another way is, well, what if I can do everything in parallel? You mention all the activities in the project are independent, one will do it in parlor. Well, you can do it very quickly this way, but do you have enough manpower to do it? And by the way, there’s still the risk that even though you did each activity individually, you remember the required behavior is the integration. So you still have to integrate it. If you work things really in parallel, there’s no going to the integration is actually going to work, even though each component on its own actually works. Get it?
Kirill Eremenko: 00:45:12
Mm-hmm (affirmative).
Juval Löwy: 00:45:13
So you have to work the integration into the project. And so if you start thinking about it, you realize and this is not my observation, but I’ll give you my observation in a second, you can model any project as a network of activities and the network can have different topology of things to do. And then on that topology you multiplex the people to do it with. And even that multiplexing has commonalities of certain things you should do, certain things you should never do, best practices and so on. And you can actually calculate in a very engineered manner, the fastest way of building the system or the cheapest way of building the system or the best combination of time and cost, and even come up with a whole list of all this thing, which are actually impossible.
Juval Löwy: 00:45:57
So suppose you have a certain project and this is really a year project, but the boss gives you six months. You know what? No amount of energy and willpower we’ll do this project in six months, if it’s really a year project. So how about if you can prove using numbers that this cannot be done in six months, that it needs at least 12 and it’s best if you give it 14 or whatever? So these are all things which have to do with designing the project. Now, in the software industry people have completely given up on this. It’s like if they’re giving up on design, they certainly have given up on designing the project and they completely missed the point that you can design projects. But that took it to another level because I said, it’s not just about time and cost. There’s another element in which is risk. Because maybe I can afford doing this project and I have enough time for doing it, but doing it this way as risky. Then that’s probably not a good idea.
Juval Löwy: 00:46:55
And in fact, most people make decisions based on risk, they don’t make decision based on time and cost at all. Actually Daniel Kahneman got a Nobel prize in economics for demonstrating that people only do decision based on risk, not based on expected utility of time and cost. And so I said to myself, is there a way for me to quantify it? Because I’m an engineer, I’m an old school engineer and only look at numbers. Is there a way for me to quantify the risk of any particular way of doing a project? Now, if I mentioned to you that the project is always a network of activities, there is perhaps some kind of a critical pass zigzag through this network, which is activities I have to do back to back, back to back, back to back, back to back, or I’m going to miss my commitments. But everything that’s not on the critical pass I have some leeway it’s called float.
Juval Löwy: 00:47:44
I can defer doing it by a week or two, maybe a month and still be okay. And if you think about it, the more float the project has the cost activities, the less risky the project is. You have more degrees of freedom. And the more critical, the more tight it is, if it’s a fully critical project, anybody sneezes, the project is late, right? That’s basically what’s going on. And so I started looking for ways of quantifying the risk of a particular project design option. You understand?
Kirill Eremenko: 00:48:14
Yeah.
Juval Löwy: 00:48:14
And I came up with, by now, I have, depending how you count six or 12 different risk models. And in my book, I only show actually two or three. But once you realize it’s possible, you can come up with new ideas all the time. And so then I said, if it’s fundamentally a three-dimensional problem, it’s got time and cost and risk. You can’t just look at two items out of it, like time and cost. You have to add the third element always, right? And so it’s like a three dimensional problem. And I started mapping it to the three dimensional space of time and cost and risk. And I started looking for patterns in the behavior of time and cost and risk in projects. You understand?
Kirill Eremenko: 00:48:53
Mm-hmm (affirmative).
Juval Löwy: 00:48:54
And guess what? Certain numbers and behaviors and commonalities emerged. And I said, ‘Okay, I’d love to invent that within any project, given that I know that this is how it behaves. Let me try and see what it takes to make this project come to the sweet spot that it should actually behave. And I’m going to stay away from all the desks on all the things that I know should never actually work.” And if you look at it this way, then before anybody writes the first line of code, you have a very nice project design with low risk, which is the cheapest and fastest way of building the project. And it’s like, what’s not to like? And it’s not applying this on software system.
Juval Löwy: 00:49:28
And even if you have a clunky design, but you can meet your commitments and the deadline you said to the customer and the cost of the customer wants to pay and you don’t mess it up because the risk is acceptable, they’re going to come to you for the next project as well. So even though when people say I design I think architecture and such, I can tell you, the secret sauce of what we do is actually designing the project. It’s not designing the system and the reason is they are not equally weighted. Because if I give you an option, suppose I have world-class architecture, amazing architecture, everything’s encapsulated, all the volatilities, all the layers, but I don’t give enough time and money to build the project?
Juval Löwy: 00:50:08
Or I give you another project where the architect is sort of clunky. It’s got some gray areas. Somethings completely blew it, but enough time and money to build it. Which one would you choose?
Kirill Eremenko: 00:50:19
The second one.
Juval Löwy: 00:50:19
Of course, the second one, right? It’s not even linear. And going back to my own carrier, I can tell you that I was the chief architect of a fortune 100 company in Silicon Valley before I was 30 years old. And this is in the most competitive place in the world for out industry. Where even the mediocre developer are pretty good if you know what I mean. What can you do? It’s very ruthless there. And so the question is why? Is it because I was the world best architect? Well, a few years later, Microsoft actually said that I’m a software legend as far as my design research.
Kirill Eremenko: 00:51:02
I heard that I heard they, they gave you this software legend title which is interesting.
Juval Löwy: 00:51:06
And they said that “You’re the number one architect in the world.” That may or may not be true. Suppose I’m number 100. Nobody argues, I’m pretty good at this. Okay. Let’s just, okay. Now the question to you is, do you think I had to be the world’s best architect or just produce good enough design in order to succeed? And the problem is managers, management, board members, customers cannot tell good design from bad design ever. You can show them orange juice and good designers they’ll say, “Oh, well,” it looks like, I don’t know. It turns out that management would never promote you or reward you based on things they don’t understand. You get it?
Kirill Eremenko: 00:51:45
Yes.
Juval Löwy: 00:51:46
And so even if I were to produce the world’s best design, every time nobody could tell the difference. Now, given the fact that they keep allowing horrible designs all the time, that definitely it’s not it. I rose through the ranks because I always bundled system design with project design. And this combination is natural glitzy and it’s explosive. And you keep sharing to management “Here’s what it will take, here is.” Nobody’s going to argue with you. Nobody. Okay? What you did this? I’m going to give you this. So out of the gate, you get the ingredient for success. And in fact management starts fighting for you and say, “Okay. You need this? I’ll get you this.”
Juval Löwy: 00:52:23
Now, if you don’t come up with project design, then you’ll do it with the developers you have. Why? Because that’s what you have. There’s no coalition by the way between developers you have and what it really takes to succeed. Now I remember this whole thing is feeding off, having the collect architecture and all that mental machinery and all the components. And so I already know all the pieces fit together. So my project design is already very much aligned with the way I’m going to build the system anyway. And so now all I’m doing is I’m doing this fine tuning of, is it two developers here? Two developers over there. That’s all I’m doing. And I can do that very quickly. And then when you start delivering on it and you always meet your commitments, you become trustworthy. And guess who they’re going to promote? Are they going to promote somebody they don’t trust or somebody they do trust?
Juval Löwy: 00:53:04
Are they going to promote somebody who just wastes money and complaints about technical debt and the previous developers, of course they are the one who messed it up not me? Or the one that does it the cheapest, the quickest, and keeps delivering on the same commitment? Guess what? Okay. So this ideas tend to turbo charge your career like you wouldn’t believe. And then it has these ripple effects because most projects are done poorly, but your projects are done right. And it doesn’t have to be even perfect because there’s a saying in the land of the blind, that one eye guy is king.
Juval Löwy: 00:53:41
Now, all the things that I do give you a second good eye and a telescope. But let’s just talk having one good eye. Everybody else is blind. They come to you and they say “How come it’s working for you, it’s not working for me?” Now you show them how to do it correctly. It doesn’t take much. Their stuff starts doing like, and then the measurement says, “Hmm, you know the higher we put this guy, the better off everybody’s going to be.” That’s more upwind for your career. And so you start creating these positive feedback loops with your career.
Juval Löwy: 00:54:14
And so that’s kind of how I just describe the first 10 years of my own career. And again, it was a journey. It wasn’t as mature as I’m showing you here, because it took me a long time to realize these things. And I kept reviewing it. I kept visiting all systems and checking up how did they do years later and what these and what that. And so eventually, about three years ago, I started writing a book on it. I take it back 10 years ago, I started writing a book on it and I wrote about half the book. And then I said, it’s going to sound crazy. I said, “Two things are happening now. One, the world is not ready for it because the world is effectuated with doing it wrong. They all think that plucking user stories off the Kalman board is the best thing there is.” And I’m not saying don’t use Agile. I’m saying don’t use Agile this way.
Juval Löwy: 00:55:08
And the other thing I said 10 years is not enough. The only test for architecture is time. Architecture needs to withstand time. And so I said, I want to see how 20 years looked like. And by the time I started getting reports on 20 years, I said, “Okay, I’m pretty sure this works.” Because most people when they write books about new ideas, it’s just suggestion. And I’m not sure that it works. I wanted to have the proofs. And I wanted to have to say, “Yes, this works for 20 years.” And in fact, every chart I have in the book is real. It’s from real project. And some of the charts go back to the 90s. Because I wanted to capture all of this in the book in a very structured way, the methodology. Here’s why it works. Here’s how you need to do it.
Juval Löwy: 00:55:53
Now, I’m not giving them silver bullet. I’m not saying ‘If you do it my way, you’re going to succeed.” That’s what quacks say. Quacks always say, ‘if you want to just do analysis this way or gather data this way or Agile this way you will succeed.” That’s what quack say. That’s such snake oil. In fact, on the very first few pages of the book, I’m saying “This book will require you a lot of work and becoming good in anything is a journey.” You know that right? Are you proud of the first program you ever wrote? How about the second one? Probably not very much. And so part is putting the effort to become good at it, but the biggest challenge is here. Working correctly and knowing how to structure systems and project correctly, requires you to rewire your brain, to think about these problems correctly.
Juval Löwy: 00:56:42
And if you have years of doing it wrong, people find it very, very difficult to change their mind. To find better ways of doing something right. Peoples are creatures of habit, right? Which goes back to what we discussed in the pre-interview that you only do things if it’s part of your habit, part of your routine, right? You only exercise if it’s part of your routine, that’s the only way you exercise. So that’s what is a challenge in my teaching is changing yourself and rewiring your brain.
Kirill Eremenko: 00:57:14
The psychology behind it.
Juval Löwy: 00:57:15
The psychology behind it. And in fact in the book, I even give some advices on psychology. Most people by the way, when they’re exposed to my ideas, they try and do two mistakes. One is they try and pontificate. They try and argue.
Kirill Eremenko: 00:57:30
What does that mean?
Juval Löwy: 00:57:31
Like a preacher pontificating, “Oh, you should do it this way. You are sinners, this and that.” And the reason they’re doing it is because they found a new belief. They’ve found a new [inaudible 00:57:43]. And that’s not a good idea. And the reason is it’s near next to impossible to argue with people. Have you ever won an argument with somebody on anything? On politics, religion, technology? You can’t argue about things. And if you start arguing you’re creating conflict and people just shut off. It’s a lot easier to do than it is to argue. Don’t spend the time arguing. Don’t spend the time educating, just work correctly, let your results speak for themselves.
Juval Löwy: 00:58:15
That’s going to be the best persuasion you can have, right? Because nobody can argue with results, literally there’s a saying, nobody can argue with results. Now they can argue with ideas. “I don’t like your ideas.” But no way they can say, “I don’t like your result. Your system it was so quick and so cheap to build and nobody complains, yeah I don’t like that.” Nobody’s going to say that. So do it right. And then you also get the credentials to say, “Yes, I know it will work correctly. You should listen to me. Look at my track record.” Okay.
Juval Löwy: 00:58:42
So the one mistake in the psychology is that people try and argue instead of doing. It’s a lot easier to do than it is to argue. The second mistake is that people try and overdo my ideas. And there’s a saying, “Learn to walk before you can run.” So if you look at the ideas I discussed with you today, it’s also a spectrum of how much of it you want to adopt, and to what extent, right? And so learning to walk before you can run is even a normal project design solution that just has enough time and money for doing it, is probably good enough for success. Now it’s a far cry from the cheapest and the fastest. Which is a lot better, but how bad it is to find something which is doable? Start with that practice doing it this, and eventually you work your way into doing other things.
Juval Löwy: 00:59:36
The same is true by the way with architecture. One of the things that I explain is that you need to come up with the smallest set of building blocks, that allows you to do these things. Because even though design is always the aggregate of those components, you want to minimize a set of components. If I can do it with 10 it’s better than 12. Why? Because we’re in the business of doing less work, not more work. Okay, now suppose you can come up with 14? It’s probably not worth the effort of spending another four months reducing it at 12 because it’s less than a difference in doing it in 14. It doesn’t matter. All it has to be is good enough. And for technical people, the concept of good enough is really hard. It’s hard for them to understand that good enough is by definition good enough.
Juval Löwy: 01:00:22
Good enough doesn’t mean it’s best. There’s best. It’s best is far after. Good enough is here. Now most people do it worst. Worst is easy. Everybody does worst. But that’s the first stepping stone of good enough. And then we can have good and then we can have best. And it’s true with anything. It doesn’t matter if you’re a doctor or a lawyer or a pilot, any professional didn’t become very good on day one. Now they’re very good. Decades later, but none at the beginning. In fact, if you’re very good at something at the beginning, you probably shouldn’t do it. And I know it sounds strange, but the reason is anything that enables you to become very, very good with no effort has no value.
Juval Löwy: 01:01:06 You only need to do for a living things which are hard. Things which people willing to pay you to add value to. If it’s very easy to do, you’re not going to make a lot of money. That’s minimum wage. A guy flipping burgers, I can train everybody to flip burgers very, very good, but I’m not going to get paid very much. So you want to stay from the low added value activities and gravitate towards the high hard added value activities, that’s a good career tip. But getting there is of course a journey.
Kirill Eremenko: 01:01:37
Wow. Fantastic, mind blown. That is such a cool ex course. Thank you very much. I can’t believe we’re already an hour into this podcast. It’s been crazy. You still haven’t called, a named the book.
Juval Löwy: 01:01:55
The book is called Righting Software spelled R-I-G-H-T-I-N-G. Meaning Righting like standing up correctly, like fixing. And Righting Software, the book aims at fixing the broken industry. The software industry is fundamentally broken at every conceivable aspect of cost, schedule, staffers, requirement, risk. Exactly. Everything is broken. And the nature of progress is that progress is always a step function. We don’t improve a lot incrementally. One day we’re here and then the next day we’re at a completely different level of existence because we did something differently. For example, the most important medical procedure of all time was washing hands. Washing hands has saved more life than anything else. And it took all the way until the 1800s people to realize to wash hands. And that changed everything. And the most important drug in medicine is penicillin.
Juval Löwy: 01:02:59
No drug counts as much as penicillin but it took all the way to the 1940s. So the book is kind of like the penicillin. It doesn’t take much, it does take a change. You have to start washing hands and do things correctly, but it’s a fundamental transformation of the career of those involved and anything that they touch from now on as long as they decide to right their career and right the industry. And I wrote the book with the objective of fixing the industry. Which as we discussed, there’s some serious things that needs fixing here.
Kirill Eremenko: 01:03:36
Wow. Wow. I am speechless. I’m just still processing all this stuff that you gave me today. It’ll take me probably a few days to get accustomed of this. I just want to say that even though your experience comes from the world of software, right? So what you carried from the 90s and what you thought about and what you came up with. We are moving into a world where data science is no longer, just like a function on its own. Still, yes there are in many cases, people can get away with just doing insights, and getting results. But more and more data scientists are required to work with developers to put their models into production and all of this can apply.
Juval Löwy: 01:04:33
And if you think about it, we’re moving very quickly to a world where the data science itself is going to get commoditized into this pluggable components which we can plug into systems. But if the ecosystem of where you plug your data components, the data science components is rotting, by extension your stuff is going to get rotten too. There’s no escaping it. You have to adopt a system perspective. Everything has to be done correctly. For want of a nail, the kingdom failed. You can’t just say, ‘Okay, my stuff is gorgeous. Everything else sucks.” No, your stuff sucks now too.
Kirill Eremenko: 01:05:07
I love it. Yeah, that’s true. That’s true. And then on the flip side, I was talking to the person who runs the TensorFlow YouTube channel. So Google has TensorFlow for artificial intelligence and, Lawrence Moroney, runs their YouTube channel. And he said their mission is to take the world of developers and help them get into artificial intelligence and data science. So these worlds are rapidly coming closer and becoming more integrated. So I want to thank you for sharing all these insights today, because they will surely help the people who are open to incorporating this. And especially I’m excited for data scientists who are just starting out into the world of developing, because as you said, bad habits die hard. So if they don’t have the bad habits, they can learn the right way right away.
Juval Löwy: 01:06:04
This is so true when I’m doing my master classes, it’s a lot easier on the junior guys that don’t have years of bad habits. And there’s always a group of people in the masterclass banging their head on the desk saying, “How come I didn’t take this class 10 or 15 years ago?” There’s always a group like that. So I agree that there’s no reason to spend years in agony, just start doing things right.
Kirill Eremenko: 01:06:27
Fantastic. Fantastic. Thank you very much. Righting Software is a book where to find out more. Juval, before I let you go can you please tell us where people can find you, get in touch, follow your career, maybe get more information on the things that you share in the internet?
Juval Löwy: 01:06:46
So the best is idesign.net and you can also try and find me at conferences I speak at least as long as the Corona is going on, I’m doing it virtually, but I’m probably going to hit the road again. I do quite a few conferences all over the world. So you can try and catch me there. And of course the masterclasses, and we will be planning two master classes in 2021. Now, every master class I’ve conducted over the last 20 years was sold out with a waiting list. And that’s normal. And then in 2020, we didn’t do a class. And so in 2021, we have all the pent up demand class 2021. So we are almost full now for 2021. And what I propose people do is contact Idesign if they’re interested in getting on the waiting list for 2022. I know it sounds crazy, but if you want to get into those things, that’s what you need to do.
Kirill Eremenko: 01:07:43
Okay. Got you. Juval one last question. What’s a book you would like to recommend? I know we already talked about your book, Righting Software. What’s another book that’s impacted your life?
Juval Löwy: 01:07:51
So the books that I really liked and which I hope would find the value with the audience here is Antifragile by Nicholas Nassim Taleb. And Taleb is not a software engineer, but he used to be a financial trader. He’s now a scholar. And the book describes the mindset you need as you build these long lasting high value systems. And most people completely get the point of Antifragile incorrectly. They think that Antifragile is robust and that’s not it at all. Now, if you think about what I discussed today, I mean you have to read the book to understand why it’s actually the same concept just presented differently. And lots of interesting tips for life as well. He’s also a great author. So it’s just a pleasure to read.
Kirill Eremenko: 01:08:39
Awesome. Thank you. Thank you. So two books we’re talked about today, Antifragile and Righting Software, R-I-G-H-T-I-N-G. Juval, thank you very much for your time today, it’s been a pleasure and frankly, an honor. I was not expecting all of this compressed mentorship in an hour. It’s been crazy good. Thank you very much.
Juval Löwy: 01:09:02
Excellent Kirill. Thank you.
Kirill Eremenko: 01:09:09
So there you have it, everybody. Hope you enjoyed this podcast as much as I did. Mind blown, right? Mind blown. So many useful, amazing, unique insights that I don’t think is possible to find anywhere else. And yeah, just so much cool stuff. My favorite part was how Juval opened my eyes to technical debt. That not all projects have to have technical debt. It is incredibly inspiring to hear that he’s got systems that he designed 10, 20 years ago that are still running and are easily fixable because of the volatility components and how he breaks them down into components.
Kirill Eremenko: 01:09:58
I can’t imagine I didn’t know about all o
f this stuff yesterday. And it must be like a huge assistance to people designing systems whether it’s in data science or just in software design on its own, incredibly valuable information. Once again, the book is called Righting Software, which is spelled R-I-G-H-T-I-N-G as in like righting without the W basically, Righting Software. Fantastic podcast. Huge thank you to Juval. I’m sure everybody got valuable takeaways from here. And as usual you can find the show notes at www.superdatascience.com/419. That’s www.superdatascience.com/419, where you will find any materials mentioned on the episode, any URLs where you can get more information plus the transcript for this episode.
Kirill Eremenko: 01:10:56
If you know anybody who is in software design and system design and project design and who might benefit from this information, don’t just keep it to yourself, send it to them. It’s super easy to share. Send them the link www.superdatascience.com/419. And you might just might help somebody open their eyes to the unique philosophy, to this unique philosophy about system design, which avoids recurring tons of technical debt and makes your projects last much, much longer. And, yeah. So share this episode. Hope you enjoyed it. Huge thank you to Juval. It was a pleasure and an honor chatting with him today. And I look forward to seeing you back here next time. Until then, happy analyzing.
