Welcome to episode #099 of the SDS Podcast. Here we go!
Today's guest is AI Learner Tuatini Godard
Why would a young man quit his comfortable life and enjoyable career to „survive on benefits“ while studying a new field without any certainty as to where it may lead him? Tune in to hear Tuatini Godard's story and life philosophy.
Along the way, you will hear us discuss the wide range of tools and ideas within the space of deep learning and AI, as well as our in depth discussion about the current state of hardware technology for deep learning and where it is headed.
In this episode you will learn:
- Using the Pomodoro Technique to Improve Productivity (10:01)
- Exploring the World of Deep Learning and AI (19:25)
- The Role of Games in AI Development (23:51)
- On Only Doing What Makes You Happy (29:17)
- Mathematics in Data Science (39:04)
- Tools: TensorFlow vs PyTorch (41:44)
- Appropriate Hardware for Deep Learning (45:45)
- The Next Big Thing: Edge Computing (54:04)
Items mentioned in this podcast:
- Homo Deus: A Brief History of Tomorrow by Yuval Noah Harari
Kirill: This is episode number 99 with AI Learner Tuatini Godard.
(background music plays)
Welcome to the SuperDataScience podcast. My name is Kirill Eremenko, data science coach and lifestyle entrepreneur. And each week we bring you inspiring people and ideas to help you build your successful career in data science. Thanks for being here today and now let’s make the complex simple.
(background music plays)
Welcome ladies and gentlemen back to the SuperDataScience podcast. Today I've got a super inspiring guest on the show, Tuatini Godard. Tuatini is my friend from France, and the way we met was when Hadelin and I were on our European roadtrip. Tuatini came to our catchup there with students and it was so inspiring to hear him talk. So one thing you need to know about Tuatini is that he used to be a software engineer, an Android developer to be exact, and he actually intentionally quit his job and became unemployed just to pursue his passion in learning about AI. How inspiring is that? When I heard that story, I was so thrilled and so excited for him.
So you will hear him say in the podcast that he's actually surviving on his unemployment benefits but he's extremely happy that he has all the time that he needs to pursue his passion in learning about AI and deep learning. And the confidence that he has that the opportunities will come out of it, that he'll just learn what he's passionate about and the opportunities will come is extremely encouraging and refreshing as well.
So in this podcast, we'll discuss quite a lot of different things. We'll talk about how Tuatini structures his day since he doesn't have to go to work. What does he do, and how does he make sure that he does follow his passion in learning and he pursues that. And we'll talk about the Pomodoro time management method which he uses, and actually I was using, even though I didn't know about it, back when I was starting my educational courses.
We'll talk about what he's learning in artificial intelligence, what the recent developments are, and we'll talk about deep learning and why he's chosen to go into that path. And we'll even cover off some of the major recent technological breakthroughs in terms of GPUs and Tuatini will give us a quick excurse into that world as well.
So, all in all, a very varied podcast, lots of interesting things that we're going to discuss, and I can't wait for you to check it out. And without further ado, I bring to you Tuatini Godard.
(background music plays)
[Speaking in French]
Kirill: Oh man, that was so cool. Just for everybody, back into English, we met in Paris, right? Tuatini, when Hadelin and I were on the roadtrip. It was a pretty cool catch up.
Tuatini: Yeah, it was really amazing to have dinner with you two guys. Really interesting stories we told each other.
Kirill: Yeah. And for those of you who don't know French, that's exactly how I felt. Like how you felt just now at the start of the podcast was exactly how I felt because I know just a little bit of French, and these guys were just talking French the whole night long, right Tuatini?
Tuatini: Yeah, yeah, but actually you told me that I articulate well, so you can understand what I am saying, so I thought it would be ok!
Kirill: Oh yeah, that's right! That's right, out of all people there, I could understand you the best. It's interesting, because everybody was talking at about the same speed, but you just articulate your words so well in French that I could understand maybe like 80% of what you were saying. It was really cool.
Tuatini: Oh, nice. Thank you.
Kirill: Yeah. But anyway, welcome to the podcast. So Tuatini Godard is an AI learner. So tell us a bit about yourself, Tuatini. What do you do for your free time, and what do you do in life right now?
Tuatini: Currently I am unemployed. By choice, rest assured. And I spend all my days actually learning everything I can around data science, and especially deep learning. Most of the time, when I have to get out of my house, I go to meetups and meet some interesting people, like when I met you and Hadelin in Paris. So basically, that's it.
Kirill: And this is really interesting, because I just found out just before the podcast that you actually had a job at a startup and then you decided to quit it because you wanted to spend 100% of your time on learning. Is that right?
Tuatini: Actually, I left the company almost 1 year ago and that was a tough choice because when you are in your comfort zone, it’s really hard to tell yourself that there is a new field which is not related to what you do currently and you want to go into that field. Basically, at that time I had multiple choices.
The first one was to stay in the company. And I really liked the company and the team I had there. At the same time, I wanted to get involved in AI. So, I went to my employer and I asked him if he wanted to stop the contract I had with him and then as of today I am living on my unemployment benefits and more like surviving on my unemployment benefits than living with it. But still, as of today, I’m doing something I really like, even if it’s hard. Because there is a lot of stuff in data science that I didn’t know, like everything around mathematics, so that’s really hard for me. Still, I really like what I do today. Even if I don’t have a good salary or good co-workers, still that’s the most important to me – to be able to do what you really want.
Kirill: That’s so cool. Is this something like you’re happy doing what you’re doing, or are you building up your skills and knowledge so that later on you can either start your own business or get a very exciting job, or is this something that you want to continue doing, just having fun with deep learning on your own?
Tuatini: Actually, I want to make a living with deep learning. Because I left my job and I don’t want to spend two years learning deep learning and then going back to software engineering and development because this is what I was doing in that startup. So, yeah, as of today I’m learning everything I can. And I don’t have any clear goal, but I take opportunities as they come. I meet people. I try to get involved in open source projects if I can, and I do what I can with my free time.
Kirill: Okay, got it. Tell us a bit more about what you did before, because it sounds pretty interesting – software development. What’s your background? Where did you study and where did you work after that?
Tuatini: As for my background, I started coding at the age of 13. I got my Master’s degree at the age of 23. And at the age of 22 I went into the startup for which I worked for 2 years. And I was the lead developer for Android over there. It was like every morning you wake up and you tell yourself, “Okay, there is this bug to fix or this stuff to add,” but you are still solving the same problem, you know. So in the end, I was a bit tired of this and this led me to this decision to quit this job and go into deep learning. But during the last years, I was involved in many areas of computer science. I was a sysadmin, I worked on security systems, I worked as a backend developer, and as an Android developer of course. And nowadays I’m doing data science and deep learning.
Kirill: Gotcha. Okay, very interesting. So, tell us a bit about your day. From my own experience, I know that it’s very hard to organize yourself when you’re just working or studying from home. You don’t have to be like, you know, at 9:00 you have to be there, at 6:00 you come home. When you don’t have this schedule that’s imposed upon you, when you have to control your own time, sometimes it can be very hard. So tell us about your day. What time do you wake up and how do you go from there?
Tuatini: Yeah, sure. Actually, before leaving my old startup, what I did was to create a clear roadmap of what I have to learn. My days actually—what I do of my days is actually using a Pomodoro Technique. I don’t know if you know this technique….
Kirill: I’ve heard about it. Tell us a bit more. My brother told me about it.
Tuatini: Yeah. Actually, it’s a really dumb technique, I would say. It’s basically you take a timer, you put it for 20 minutes, and you work during these 20 minutes but you have to work, really work, like no Facebook, no eating, nothing, just work. And when these 20 minutes are over, you can start having a pause for 5 minutes and then you go back to your work for 20 minutes again. And actually, I use this technique every day to get at least 4 hours of really productive work or not work, but learning, really productive learning at least per day so that I know that I was useful, you know, that I didn’t lose my time spending my time on just social media so that I can keep up with my goals which are defined on my roadmap, you know.
Kirill: Yeah, that’s really cool. So, you use this Pomodoro Technique. For those listening out there, it’s a very interesting technique. I find it a bit controversial, or not necessarily controversial, but perpendicular to a lot of the other techniques that exist out there. For instance, there are philosophies out there that say that it’s very hard when you switch over between tasks. Like, you’re doing something for an hour or two and then you switch over to something else. It takes you at least 20 to 40 minutes to readjust your brain to work on something else.
Whereas in the Pomodoro Technique, every 20 minutes you have to get up and do something. And you can set the stamp to whatever you want: it can be 10 minutes, 20 minutes, 30 minutes. You get up and you have a break and then you come back to the technique and so on. You’re constantly consciously interrupting yourself, which is contrary to the other methodologies and philosophies where you try to limit the interruptions. It’s interesting. It’s great to meet a person like Tuatini, for whom it’s working.
Tuatini: Actually, I don’t respect the technique 100%. (Laughs) When I’m working on a piece of code and I really want to solve it, if the timer goes off, I just start it again and I just don’t take the 5-minute pause between the two 20 minutes. Actually, I just use this technique to know that at least I commit 4 hours per day of doing productive work. I would say I only use this for that, but I don’t respect it 100%. As you said, it’s really hard to shift your focus to something else for 5 minutes and then come back to the same thing you were doing before. Sometimes it doesn’t work for me either.
Kirill: Yeah, gotcha. I can totally relate to that. You kind of get into this state of flow, right? You’re coding and you’re into it and the last thing you want is to get up and have a break. I think maybe that’s the way it’s designed. If you’re getting up and having a break, that means you’re not in a state of flow, or at least you’re kind of keeping control of how you’re spending time. It’s harder to fall into the pattern of getting on Facebook or doing something irrelevant to your work that way, because even if you do that, then you’ll only be limited to 20 minutes before you snap out of it and you realize, “Okay, the next 20 minutes I will do well.” But if you do get into the state of flow, definitely don’t get out of it. Because when I feel that I’m in flow and I’m getting things done, like you say, you don’t want to get up and you just restart the timer.
Actually, I did something similar when I was also quitting my job and I was working in the evenings and on the weekends, I was valuing every single minute. Even without knowing the Pomodoro Technique, what I did was I broke my hour into four parts, so 15-minute parts, and I set a timer for 15 minutes. And my goal wasn’t to get up and go have a break.
My goal was to reset the timer every 15 minutes, but on the side of my desk I had a paper and a pen, in my notebook I had a page dedicated to this, and it would be split into two parts. On the left it would be ‘productive,’ on the right it would just say ‘non-productive.’ And then whenever the timer would go off, I would reset it and I would just put another point or a line into ‘productive’ or ‘non-productive.’ So, I would look back and I would say, “Okay, were my 15 minutes productive or did I just waste them?” And then I would just put a line—you know how they do, you put like four lines and then you cross them over and that’s five and so on.
I put a counter into one of the two sides of the page and that way, at the end of the day, I could look back and similar to you I would be like, “Okay, I’m going to count how many 15 minutes I spent wasted and how many 15 minutes I spent productive,” and based on that, I would come up with it and I would be like, “Oh, I actually had 4 hours or 8 hours of actual productive work this day,” and I’d be satisfied with that. If there was too much unproductive time, I would have to look back at it and understand what I was doing wrong and the next day it would become better.
Tuatini: Yeah. Actually, I also used, until a certain time, I used a Don't Break the Chain Technique. I don’t know if you know about that.
Tuatini: It’s actually also very dumb, but it’s like—for example, if I tell myself that I have to commit 4 hours a day of really productive work, I have to keep up with that for the rest of the days. I can say that on weekends I don’t have to respect that, but if I have to respect that from Monday to Friday, I have a calendar and I have to check each day that I didn’t break the chain, you know, I didn’t commit only 3 hours on one particular day or things like that. It’s just that, actually. And I had this calendar where I checked every day for like 3 months, and then I went on a trip and everything was screwed, you know. (Laughs)
Kirill: Yeah, I can imagine. But that’s good. That’s how you get stuff done, right? I’m sure you’re learning a lot.
Tuatini: Yeah, sure.
Kirill: We’ll get to that in a second. So you wake up and you start working right away, or do you go for a walk? Walk us through your day. What do you usually do first?
Tuatini: Actually, as soon as I wake up, I do my work. And if I see that I’m reaching my 4 hours on the day, I can try to do some stuff which are not useful, like browsing the Internet or—I don’t know, but nothing special about it. Basically all my days look like that. I wake up, I work, I commit to my 4 hours and then I can play video games or talk to my girlfriend or—I don’t know.
Kirill: Gotcha, have a social life. But why only 4 hours? Is there a reason? Why not 7? Why not 8? Why not 10?
Tuatini: Actually, it’s not only 4 hours; it’s at least 4 hours. So, it’s that I know I was committed to these 4 hours. Yesterday, I think maybe I spent 8 or 9 hours because I was in a Kaggle competition and I was like, “Why doesn’t my accuracy go up?” So, I was on my code and I just couldn’t stop.
Kirill: Gotcha. So, basically it’s better to set a reasonable but not a very high level that’s easy to achieve and you are always satisfied with your day and if you do more, you do more.
Tuatini: Yeah, exactly.
Kirill: Gotcha. Okay, really cool. Okay, so that’s how your day is structured. Now tell us a bit about what is it that you’re learning. We’ve had a few guests talk about deep learning on the podcast, but what’s your version of deep learning, your summary on deep learning in a few sentences? What is it?
Tuatini: Currently I’m learning everything I can around CNN and computer vision. I just finished the courses on phase.ai, and I plan to start the new one you released on Kickstarter, you know, as soon as you release it. I still didn’t finish the one you released on AI because I was doing other MOOC before that. But yeah, I think I will probably base my work around computer vision and reinforcement learning because these are the two fields which fascinate me the most.
Kirill: That’s really cool. Why do you like computer vision so much?
Tuatini: Well, compared to NLP, you have this feedback, this visual feedback, so it’s much more rewarding than NLP. I don’t know how to—
Kirill: Yeah, yeah, I got you. NLP, for those listening, is natural language processing. Is that right?
Tuatini: Yeah, sure.
Kirill: No, I totally understand. It’s more gratifying to see your results. You can actually see green boxes and stuff like that, what’s going on.
Tuatini: Yeah. And there is also the fact that the field of computer vision opens so much more opportunities than NLP, you know. When you are doing NLP, in my opinion at least, I don’t see that many opportunities compared to what you can do with computer vision. With computer vision, for example, you can make robots understand the world, you know. But with NLP, you try to make them understand our language, which is not an especially real representation of our world. It’s a language which was created by humans, whereas computer vision is a thing which is native. I don’t know how to say that, but you get me.
Kirill: Yeah, gotcha. It’s just like information that’s out there regardless of language.
Tuatini: Yeah, exactly. It was not made by humans, if you will.
Kirill: Yeah. Okay, very interesting. And you’ve got a very interesting project that you’re working on or you’re planning on working on – StarCraft, right? Tell us a bit about that.
Tuatini: Yeah. I didn’t start yet, but I plan to start hacking around with StarCraft and really start diving deep into reinforcement learning because this is actually one of the fields which is very—not a lot of startups are involved in reinforcement learning except OpenAI and DeepMind, and I think that’s because it still has to make its proof. Because as for today, with CNN and RNN – convolutional neural networks or computer vision, and RNN, natural language processing – you can already achieve a lot of real work. But with reinforcement learning, it’s much more complicated because you cannot—I don’t know how to say that, but it’s much harder to see the goal of what you can do with reinforcement learning.
Kirill: Gotcha. Just so that everybody is up to speed, let’s break it down because there are people listening to this podcast, maybe this is their first time encountering reinforcement learning or CNN and RNN, so I’ll just explain the abbreviations. CNN is convolutional neural network, it’s a type of deep neural network that is used to recognize images and process them and recognize objects and images and so on. RNN is a recurrent neural network. It’s a deep neural network that has memory. That’s what’s used for natural language processing because like words and sentences—you need to know what words came before the current word to make sense of the meaning. And reinforcement learning is a technique which—maybe you can explain reinforcement learning, Tuatini. What’s your explanation of reinforcement learning?
Tuatini: Actually, for me, reinforcement learning happens when you have a goal. For example, let’s say you take a game—for example, let’s take “Mario” because everyone knows “Mario.” Your goal in “Mario” is actually to go to the end of the road and get the flag at the end and you win. So, in reinforcement learning, you define some specific goals that you want your algorithm, or as you say in reinforcement learning, your agent, to reach. But you don’t tell him about how to play “Mario.” You just show him, for example, people playing “Mario,” and it just figures out how to play “Mario” based on how the human players were playing it. Basically that’s it. But I believe there’s a lot more things you can do with reinforcement learning, apart from games, but as of today, I don’t yet have this vision because I didn’t look deeply into reinforcement learning yet, so I can’t really tell.
Kirill: Yeah, exactly. There’s lots of applications. Reinforcement learning basically allows computers and robots to learn from the feedback they’ve been getting from their environment or, as Tuatini mentioned, from observing humans and others play games. It’s a way for computers to learn without us hardcoding the algorithms into them, without saying, “If this, do that; else, do this.” You know, without the hardcoded algorithm, they can still learn.
So, the very interesting developments in the world these days are using reinforcement learning for very complex tasks. And games are a cool environment to test this stuff out because games represent a mini real world with their own rules and so on. So if you can make a robot or an AI work in a game, that means you’re one step closer to making it work in the real world. And one of the recent breakthroughs was—I think it was OpenAI that created an artificial intelligence that can play the game of DOTA. Did you hear about that?
Tuatini: Yeah, I saw that. Actually, some people said on Reddit that it was kind of dumb because it took a path in the game that the algorithm wasn’t aware of. It was just failing at completing its tasks. I don’t know, I didn’t look further into that, but—
Kirill: Okay. It’s definitely getting closer. This game is quite a complex game where they have annual competitions and the pool prize is like $2 million or $8 million, it’s growing every year. This artificial intelligence can already play comparable to a one-on-one player. The next step is five-on-five and so on. Those who are interested can check it out online, it’s called DOTA – I think it stands for “Defence of the Alliance” – version 2.0 – and artificial intelligence is playing that game. You can have a look at that. So you’re venturing into an even more complex territory. You want to create an AI or take on the challenge of exploring how to create a reinforcement learning algorithm for StarCraft. StarCraft is a different type of game. It’s a strategy game, right?
Tuatini: Yeah. Actually, as I understood, the most complicated part in this game is that when you start the game, you are not aware of the map of the game. You actually have to do some discovery by sending what they call in the game a probe, so you’ll just send the probe to try to ‘map the map,’ you know. But this is really challenging because when an AI is not aware of its whole environment, it makes things more complicated, you know.
Kirill: Yeah. That is going to be very interesting to see, how you take on that challenge. So, once you dabble on with the game—you mentioned you don’t have any ideas on how to apply this in real life, but how are you going to go about that? I think that’s an interesting question. Let’s say you’re successful at creating an artificial intelligence to play StarCraft. What is your next step for turning these skills that you’ve developed into a business, or into something that you can bring to the world and add value to the world in the real world?
Tuatini: To be honest, I don’t know. (Laughs) All I know is that I want to work on something that I like. And this is the very reason why I left my old startup, you know. The future will tell me. I’m the kind of guy who likes to do what he wants to do, but I do not worry about the future like, “Will I start a startup with that?” or “Will I work in that field with big companies?” I’m only worried about doing the stuff I like, you know. And random events of life will happen. Maybe I will have some opportunities to meet people who are also interested in that field and who have ideas to make that into a business. I’m not close to anything, but as for now, I don’t know. I can’t tell you how I could apply it to a business.
But as you told me when you came to Paris, there is actually one real application for reinforcement learning that Google made, which was to reduce their cost in electricity, and it was reinforcement learning. So I believe that you can make very useful stuff, but as of today, I’m not aware of what you can really do, what is the potential of reinforcement learning. I cannot tell you that I want to create a business out of that, you know.
Kirill: No, I totally understand that. And that’s really inspiring, right? It’s so cool to have the courage to just say, “No, I don’t care what everybody else thinks. I don’t care that I don’t have a job. I’m just going to do what I love. And I don’t even know where it’s going. I don’t know what opportunities I have in the future, but I know they will come. I know that if I’m doing what I love, then the world cannot happen in a way that I will not have those opportunities. They will come no matter what. I’m just going to keep doing what I’m doing and when they come, they come.” That is very inspiring.
Tuatini: Yeah. Well, actually this is all I did for my whole life, you know, because I was born in French Polynesia and I lived on a very small island. And I really liked computer science so I was just committed to it. I didn’t have really good grades in school, but I didn’t care. I was only passionate about what was useful to me and what I wanted to do. And then when I came to France, I started applying this knowledge.
So, I don’t know if it’s the randomness of events or if it’s chance, but I believe that you cannot have guarantees in the future of what you will do. For example, I cannot tell myself that as of today I will work on data science for marketing so that I’m sure that I will get a job in that field. For me, I just want to work on things I like. And at some point, you will find something according to that, you know.
Kirill: Yeah. And how good is that, that when you do find something, because all the time you’ve been doing just the things that you like, you’re guaranteed that you’re going to like it because you’ve never even thought of doing anything that you don’t like?
Tuatini: Yeah, it’s really rewarding when you find something. It’s the way I live, actually. (Laughs) Actually, I think randomness of events can also come, because I told you I’m surviving on unemployment benefits and actually the guy who gives me these unemployment benefits can just come to my door and say, “We will cut the grass under your feet.” So from one day to another, I can find myself not having any income but I just tell myself that when the time comes I will just figure out what to do.
Kirill: Yeah. So you’re not stressing out about it? It’s not a problem now right? “It might happen, it might not happen. If it happens, then that’s when it will become a problem and that’s when I’ll sort it out.”
Tuatini: Yeah, exactly.
Kirill: Very interesting. I’m actually reading a book now by Eckhart Tolle, it’s called “The Power of Now” and there he talks exactly about that. It’s very interesting how your story depicts the philosophy that he explained. He says that “There are no problems in the world. Think about right now. Right now, do you have a problem? Does anybody have a problem? Not five minutes from now, not a day from now, not a year from now, but right now do you have a problem?” No, all you have is the present moment and you have the situation that you’re in. So, what we perceive as problems, somebody else in your situation might say, “Oh, I might lose unemployment benefits. This is a problem.”
But what we perceive as problems is actually a situation that might arise in the future. And when the future comes, when it becomes a present moment, if it arises it’s not a problem, it’s a situation you’ve got to deal with. “It’s a situation I’m in. I’m going to deal with it. Okay, done with the situation. Now I’ll keep going with my life.”
Tuatini: Yeah, exactly.
Kirill: It’s a very healthy mentality, right? It helps you be calm.
Tuatini: Well, I would like to tell you that when this day arises, I won’t stress a bit, you know. (Laughs) But I think I will figure out what to do. There is no such thing in life as insurance. You can’t say that, for example, “Oh, I’m great in my position at the company where I’m in and I want to work for the coming 10 years,” because you don’t know what will happen. Maybe you will never get a salary raise, maybe the company will fail or maybe you will get hit by a car and die. I don’t know. So there is no such thing as trying to forecast your future in terms of insurance, job insurance, for example.
Kirill: Yeah. I totally agree. You’ve got to be not even prepared, but accepting of whatever comes. Whatever comes, comes, and we will deal with it when it comes.
Kirill: Cool. That’s very inspiring. I’m sure a lot of people listening to this will have something to think about. Tell us, has somebody inspired you? Has there been like a mentor, a life-changing event, something in your life that has inspired you to choose the path that you’ve chosen of software development and then get into deep learning and so on? What’s been the most inspirational thing for you?
Tuatini: For software development there were two things. The first one is that I wanted to get out of my little island. (Laughs) And the second one is still related to that. My father told me, “What do you want to do? You want to watch the cows and do agriculture? Or you want to go abroad and make relevant things?” So, I quickly made my choice.
As for the influencer, I don’t have any person who influenced me. Well, not one that I remember. I think it’s a whole, you know. I saw a lot of people in the field of AI and I was always passionate about creating AI when I was young, but at that time I was afraid of math and there wasn’t really a lot of things happening in that field, [indecipherable], as everyone called it. And when I saw all these views coming from TechCrunch and your MOOC, for example, all the resources online to learn about AI actually, I told myself, “Yeah, let’s go for it.”
Kirill: So you could see the potential?
Tuatini: Yeah, exactly. But there wasn’t just one thing that led me to that decision, you know. This is just a whole, I would say.
Kirill: Okay. And was it hard to start? There’s probably a lot of listeners who are into data science but not yet into the field of deep learning or artificial intelligence, and they can also hear about it, hear about the potential and see things, how it’s changing the world, but it’s very scary. There’s a lot of programming, there’s a lot of complicated libraries and even mathematics and so on. So was it hard to start when you made that decision?
Tuatini: Actually, yes. (Laughs) As I told you, I have a software engineer background, so it’s really like my comfort zone when we are dealing with computer science stuff, you know, whether it be sysadmin or programming or anything. But when it comes to math, that’s a completely different story. (Laughs) Like, as of today, when I look at a math formula, when I look at one on paper, I just want to go on a tutorial, or find an alternative to implement what I want to implement.
But this past week, I was trying to implement the Unet paper. Unet is actually a technique in deep learning to segment images. So basically, let’s say you have a car in the middle of the image and you have a background. You just want to get the car and have a transparent background behind it, you know. Unet is meant for that. So what I’m trying to do is to implement the paper from A to Z, as you used to say, and really look at the math formula and try to make sense of that. Because I think what is really hard for me in the math is not the logic, but it’s actually like a foreign language to me.
Like, if I see some sigma sign, I know it translates into a ‘for’ loop in Python or things like that, but there is a lot of signs that I don’t know in math. It’s just really foreign to me. But I’m working towards that goal. And my goal actually is to be able to take a paper, implement it and ship it into production. Basically to do the whole workflow. Not just implement the paper or just ship things into production, but be able to do the whole thing, you know. Because if someday I will have to create my business or contribute to something, I will have to deal with this whole process. You won’t be constrained just to implementing the paper or just shipping things into production.
Kirill: Yeah, I totally agree. That’s a very big step, learning math. I agree, if you want to do the full flow of things, then you’ll need to know the mathematics. At the same time, for the benefit of our listeners, I just wanted to say that there’s ways to get into this field even without delving too deep into the mathematical aspects of things, but rather just understanding intuitively how and why things work and then understanding what goals you want to achieve and then putting everything together. It’s like building Lego bricks together with the pre-coded libraries and so on. Like in PyTorch—well, less on PyTorch and more in the Google one—
Kirill: Yeah, with Keras, if you use Keras there’s a lot of stuff pre-coded for you, right?
Tuatini: Yeah, exactly.
Kirill: You can just build this AI with a few lines of code. Even if you don’t know that much coding, you just put in a few lines of code and there you go – you’ve got a convolutional neural network going or something like that.
Kirill: Do you think that’s a valid way for people to start?
Tuatini: Yeah, sure. Actually, this is how I started, so yeah, totally. If you want to start and you don’t have a lot of background into mathematics or even computer science, it’s better to work with high level libraries, as we call them, because actually it obstructs every low level computation you could do with Flow or TensorFlow or with PyTorch, for example, because Keras actually is a library on top of TensorFlow so it extracts a lot of stuff you would spend hours to figure out how to do.
Kirill: So do you use Keras or do you use PyTorch?
Tuatini: I started with Keras, but currently I’m with PyTorch. The reason is very simple because with Keras, as it’s on top of TensorFlow—TensorFlow uses what we call static graph—
Kirill: Yeah, and dynamic graph.
Tuatini: Yeah. So it basically means you have to define all your pre-processing input. You have to define how you will multiply this or that. And then when you have defined your graph basically, you start a session and you run the thing. But the drawback with that is that when you are used to work with IDE and debuggers, it’s really hard to check on things. It’s really hard to look at, for example, “If I multiply this matrix by this scalar, what is the result of that?” Actually, TensorFlow has its own debugger, but I didn’t get a chance to look at it. But the difference between TensorFlow and Keras and PyTorch is that PyTorch uses dynamic graphs so that as you go through your code you can put break points to your code and can just look at the result of your code, actually. It’s not like blackboxed as you can find in TensorFlow. That’s the main reason why I moved away from Keras, but Keras is very great to start with.
Kirill: Yeah, definitely. And it just makes things easier for people looking at this. It’s a natural progression, right? When a child is learning how to speak, they don’t start by reading Shakespeare. They start with the basics and then they progress. It makes sense. Let’s talk a bit about hardware. I think that’s a topic close to you because from our catch-up in France, I remember you were talking quite passionately about your computer setup and so on. So tell us a bit about that. For deep learning you obviously need a powerful computer. How did you go about that?
Tuatini: I started with buying a GTX 1070 because you have to get a really good GPU, especially if you’re working on a computer vision program with deep learning. With NLP it’s less relevant, because when you are inputting images into your model, images are actually big matrices, so you have to have a really good GPU to handle this kind of data. Then I bought myself a GTX 1080 Ti.
Kirill: Sorry, you lost me there. Is that a better system?
Tuatini: Yeah. It’s actually better, yeah. So, what I actually have is I have two computers: one is in my basement and one at home. And I have my MacBook also and I work on my MacBook and I actually execute my code remotely on this machine and I get the result back onto my MacBook. Actually, on my blog – I will give you the link – I teach people how to build their own rig. And I really try to explain everything, like how do you do that and what does it mean, not only what are the steps to build your rig, but also what does this step mean so when you have to fix your rig, if it’s not working anymore, you know what you are dealing with. Actually, I have—I don’t know the model of my CPU anymore, but I have a good CPU and especially a good GPU and 32 GB of RAM. I think it’s the minimum for a deep learning project.
Kirill: Gotcha. And tell us what’s the difference of CPU and GPU and why do you need GPUs for deep learning?
Tuatini: Actually, deep learning deals a lot with matrices and GPUs are a specialized kind of hardware which is meant to deal with matrices. This is why you can transfer your data to your GPU memory and then make the GPU work instead of your CPU. And the difference between the two is that—I don’t know how to say that, but GPU is really parallel, it does a lot of parallel work, it has like 3,000 cores, compared to your CPU, which has 8 cores or something like that. CPU can do a lot of stuff, but not really parallel. In contrast, the GPU is meant to deal with very specific narrow tasks, but it can parallelize a lot of stuff. So that’s the main difference between the two.
Kirill: And GPU are actually for graphics, right?
Tuatini: Yeah. Graphics, if you think of it, are actually just matrices, you know. So yeah, it’s why GPU can also be used to train deep learning models.
Kirill: Yeah. So it’s just a convenient coincidence that GPUs were originally developed for graphics, which are matrices, and “We’re going to run some deep learning now on these GPUs, only a trillion times faster?”
Kirill: And did you hear about the NVidia Volta chip?
Kirill: Tell us a bit about that, because Ben Taylor told me about it a couple of times and I’m not really up-to-date with it. What is this Volta chip and why is it so special?
Tuatini: As for today, there is only one GPU which was released with this chip, with this new architecture, which is Tesla V100. Volta is actually an architecture. Basically, you have different kinds of architectures: there is Maxwell, there is Pascal, there is Volta. Currently, with the latest models of GPU you can find on the market, we are with Pascal. And Volta actually is one step ahead, and it’s GPU which was made especially for deep learning. It’s an architecture which was made especially for deep learning because with Pascal, it was made more for gamers because it uses a special architecture, which actually has some optimization especially made for games.
But these are optimizations which aren’t needed for deep learning models, so the Volta architecture, at least the Tesla V100—it doesn’t really take away, but kind of takes this optimization and it also uses the half-precision mode which is—how could I simply explain that? Basically, you have two modes, which are FP32 and FP16. FP32 is actually to deal with numbers with really good precision. And FP16 is actually what we call half-precision, which is actually what we need in deep learning because we don’t need to have a lot of decimals in deep learning.
Kirill: Floating points, how many floating points you have. Gotcha.
Tuatini: Exactly. Volta was made actually to deal with half-precision in a way better sense than past architecture. As of today, if you try to put matrices with half-precision on your GPU, you would get really worse results than if you put FP32 precision matrices.
Kirill: So how come there’s only one of them in the world then?
Tuatini: You mean one of—
Kirill: There’s only one type, right?
Tuatini: Because it’s fairly new. And as of today, I don’t think it’s yet released to the public. I think NVidia sends it only to a few people involved in artificial intelligence, but you cannot buy this GPU yet. They will release it next year, I think. I’m not sure about that, but as of today, it’s hard to get your hands on this GPU.
Kirill: And it says that it’s about $150,000 for one of them. Crazy.
Tuatini: Yeah. (Laughs) It’s not for everyone.
Kirill: Yeah, that’s right. Okay, that’s really cool. Thanks a lot, Tuatini. I just have one question to wrap up the podcast here. You obviously have a huge experience in this field and all this knowledge that you’ve acquired in a short period of time. From what you see and from what you know about deep learning and AI or data science, about all these fields, where do you think this is all going and what should our listeners look into in order to prepare for the future?
Tuatini: Given the past events, the past news I saw passing by over the Internet on TechCrunch, Reddit and things like that, it seems like a lot of VC are interested in investing in startups which deal with edge computing. Simply put, edge computing is being able, for example, to learn your deep learning models not on the cloud, but directly on your device, for example an IoT device or your smartphone. And I feel like it’s the next step, that you are able to run directly your deep learning models onto your device locally.
Because if you take, for example, drones, when they have to map their environment and avoid obstacles, if they lost their Internet connection or if there is a lot of latency onto that connection, they cannot make the decision quickly enough to not go into the obstacle, you know.
Yeah, I saw a lot of people saying that the next step in deep learning is to be able to ship model into production, and actually even François Chollet, who is the author of Keras, said that the deep learning field in term of research paper is told and the real problem now is to be able to ship this model into production, whether it be on the cloud or on edge computing. But I feel edge computing is the next big thing for me.
Kirill: Gotcha. Okay, thanks a lot. That’s something to look into. You normally think that everything is going offline more and more now, but as you say, some applications have to make very quick decisions.
Tuatini: Yeah. And if you look at the chipmaker, for example Qualcomm, for example, or even if you look at the latest iPhone, iPhone X that Apple released—
Kirill: iPhone 8.
Tuatini: They also released iPhone X, I believe.
Kirill: Oh, okay. I didn’t know that.
Tuatini: Yeah. There’s 8, 8S and X. X is the one without any buttons on it. And actually you can do face recognition directly, locally on the phone. And when they spoke about the chip which is in the iPhone, they talked about a neuro-engine, you know, and same goes for Qualcomm. They are trying to make this kind of core processors which are meant to deal with deep learning models. So, even by looking at the hardware makers, you can see that they are trying to do this edge computing scene.
The same thing goes for Movidius. Movidius is the company which was bought by Intel. They created the neural compute stick, which you can buy for $70 and then run your deep learning model on that. But be aware that these kinds of devices are not meant to train your model, but just to run inference – in other term, predictions.
Kirill: Yeah. And to your point, if anybody wants a really vivid example of this, I’m not sure about the latest iPhones, but on the more older versions – for instance, iPhone 5 or maybe 6 and 7 as well – if you try to do speech-to-text, you know, you’re trying to type a message and you click that little microphone button on the left, and I’m pretty sure it’s the same thing on Android phones like Samsung, you click that button and it will convert what you’re saying into written text, into the text message. So you can dictate the text message. And that’s really cool. That’s actually deep learning in action. But if you switch off your internet connection, if you put it into flight mode and you try to do the same thing, it won’t work because it’s actually going through the cloud. It’s sending your voice to the cloud, it’s recognizing it there and then sending the text back to your phone.
So this is what we’re talking about here when Tuatini is saying “edge computing,” is that those models, and that hardware and everything, will be integrated into your device. That’s the challenge. In order to perform that there and then so you could, for instance, do the speech-to-text even while you’re offline.
Tuatini: Yeah. And that’s really interesting for people in this field, because actually, instead of paying cloud instances to run your deep learning models, they just let the client pay for their latest iPhone and then they ship their application which runs into the iPhone. They just cut costs regarding this.
Kirill: Yeah, gotcha. And cutting costs means competitiveness and competitive means successful business.
Kirill: Okay. Thanks a lot, Tuatini. Could you give us some details on how our listeners can find you, follow you or connect with you if they’d like to learn more about your journey?
Tuatini: Sure. There is my LinkedIn, my e-mail, and I have also a personal website where I blog from time to time, which is http://tuatini.me. The next blog post I’m going to do is, as I told you, about Unet. And actually, what I want to show is how I went from reading the paper to doing the implementation and explaining each step of the mathematical formula and translate it into code. I think that it may be useful to some people. I know some people found useful the last article I released on how to build your deep learning rig, but the next coming article will be on real deep learning stuff. As for now, there is only computer science stuff; there is no deep learning stuff.
Kirill: Gotcha. I’m looking at it now. It’s very detailed. The posts have a lot of information. So if anybody listening wants to build their own deep learning machine – and I know there’s a couple of people like that who are following this podcast – then check it out, it’s tuatini.me. And of course, we will share these details in the show notes. I’ve got one more question for you
Tuatini: What is the one book that you’d like to recommend to our listeners?
Tuatini: Regarding the book, there is one which is not really related nor to AI nor to data science, but it’s a really good book which opens your mind, I would say. It’s called “Homo Deus.” Actually, it’s made by a really famous historian that you can find on TED because he gave some talk there. He basically created two books: First one is “Homo Sapiens,” which tells the story of where we came from; and “Homo Deus” is where we are going. And he’s not only talking about artificial intelligence and stuff, but everything about our future. At the same time, he doesn’t put himself as someone who wants to predict the future. He only opens your mind to think about what happened in the past and now look at our present and imagine our future. It’s really, really interesting and I really recommend this book. I really loved it.
Kirill: Fantastic. I might check it out myself. I just googled it, it’s called “Homo Deus: A Brief History of Tomorrow” by Yuval Noah Harari. Very interesting. Once again, thank you so much, Tuatini, for coming on the show and sharing all the insights. It’s been a fantastic episode. I really loved to learn from you.
Tuatini: Thank you.
Kirill: So there you have it. That was Tuatini Godard, AI learner. A very inspiring episode! We wish Tuatini the best of luck. I’m sure that he’s going to definitely find the right application of all of these skills that he’s learning and there’s probably tons of companies that would love to have a person like Tuatini on their team, somebody so passionate about what they’re doing. And as you saw from the podcast, there were lots of things that were discussed.
My personal favourite out of all of them was just the whole commitment and dedication to his passion that Tuatini has; how he quit his job and became unemployed just to follow his passion. It takes a lot of guts, it takes a lot of courage to do that, and it’s very inspiring to see that there are people like that in the world, who can jump straight into it and know that they will be successful if they follow their passion.
And I wish the same to everybody. I think when you do what you’re passionate about, when you do what you love, then eventually you will become successful and it’s just a matter of time. So make sure to follow Tuatini on his LinkedIn to see how his career goes. I’m sure there are lots of exciting and inspiring turns that are coming up for him.
And on that note, you can get all of the show notes for this episode, the transcript, and a link to Tuatini’s LinkedIn at www.superdatascience.com/99. If you enjoyed this episode, we’d really appreciate your review or a rating on iTunes. All of your reviews definitely help us spread the word about data science. We’re getting close to our 100th episode, so we can’t wait to share more exciting and fun guests with the rest of the world. Thank you so much for being here. I can’t wait to see you here next time. Until then, happy analysing.