Kirill Eremenko: This is episode number 373 with Artificial Intelligence Advocate at Google, Laurence Moroney.
Kirill Eremenko: Welcome to the SuperDataScience Podcast. My name is Kirill Eremenko, Data Science Coach and Lifestyle Entrepreneur. And each week we bring you inspiring people and ideas to help you build your successful career in data science. Thanks for being here today, and now let’s make the complex simple.
Kirill Eremenko: Welcome back to the SuperDataScience Podcast, everybody. Super pumped to have you back here on the show. Are you ready to talk about TensorFlow? I hope you’re pumped because I am pumped. I just got off the phone with Laurence Moroney, who is the artificial intelligence advocate at Google. And in this episode, you’ll find out everything you need to know about TensorFlow in order to get going and progress your career with this amazing tool.
Kirill Eremenko: So Laurence, among other things, runs the TensorFlow channel on YouTube, which you can find out to youtube.com/TensorFlow. He’s got over 250,000 subscribers and this is a channel dedicated to helping people, especially developers, to master artificial intelligence tools and use TensorFlow to create or to develop AI solutions and create with artificial intelligence. And so this podcast is going to be extremely useful to anybody who wants to master TensorFlow, get into TensorFlow, get a certification in TensorFlow, but especially, it’s going to be very useful to developers. One of Laurence’s missions in life right now is to help the 30 million developers who are in the world to understand that they can actually use artificial intelligence. So if you’re a developer, definitely this is a great podcast for you.
irill Eremenko: So some of the things that we’ll be covering off in today’s talk, that AI is a new programming paradigm, which should and is available to all developers and all programmers out there. We talked about TensorFlow Lite, TensorFlow JS, TensorFlow Mobile, TensorFlow Extended or TensorFlow TFX and in general, the whole ecosystem around TensorFlow and that’s a great way to expand your knowledge about this ecosystem. We talked about a graph base versus eager base high level Keras models, machine learning versus programming models, math versus intuition. Laurence’s free course, which you can find on YouTube and consists of 10 videos for developers and what it can do for your career, the TensorFlow certificate exam, something absolutely new in the TensorFlow ecosystem. It was developed by Laurence and has been around for only around three months so you can be one of the first people to take this exam and you’ll learn about it in this podcast.
Kirill Eremenko: You’ll also hear about the three skills that you need in artificial intelligence in order to get a job and what these skills are and how the TensorFlow certificate exam can help you confirm that you have these skills. We’re talking about courses, AutoML and Keras tuning, the AI community in the space of TensorFlow and lots and lots of other cool topics. Can’t wait for you to check out this conversation. We had lots of fun and I’m sure you’ll pick up a lot of amazing insights from Laurence. So without further ado, I bring to you AI advocate at Google, Laurence Moroney.
Kirill Eremenko: Welcome back to the SuperDataScience Podcast, everybody. Super fun to have you back on the show and today we’ve got a very special guest, Laurence Moroney, calling in from Seattle. Laurence, welcome to the show.
Laurence Moroney: Thank you, thanks so much. I’m really happy to be here.
Kirill Eremenko: Very, very excited and honored to have you on the show and for all these amazing things you’re doing for the community. We’ve got a lot of things that I would love to talk to you about in the area of TensorFlow. Maybe to kick us off, could you give us a quick overview who is Laurence Moroney for those of people who may have not yet encountered your courses and YouTube videos?
Laurence Moroney: That’s a good question, I really don’t know. How do you answer the question, who are you? And it’s like in Les Mis, as he says, “24601.”
Kirill Eremenko: What is that?
Laurence Moroney: Les Misérables, it’s a musical and it’s like, one of the songs in it, it’s a guy that’s like, “Who am I? Who am I? Who am I?” And his convict number was 24601, so that’s what he gives as his answer. So I’m not 24601 because that’s Jean Valjean. So who am I? I’ll just talk about what I do maybe. And so right now I work at Google and so I’m the lead AI advocate at Google. And it’s my job to try and really rise the current around AI and machine learning so that really developers can begin to understand it, it can become a little bit less exclusive.
Laurence Moroney: Sometimes like a little bit of a frustration I’ve had over the years was that a lot of AI and ML stuff is very academic, it’s all about reading papers, about doing math and those kinds of things. And of course, there’s still an element of that to it because it’s very cutting edge, but in the last few years, it has really changed that, it’s become much more approachable for software developers. So I’ve kind of really made that my pivot to really help software developers understand what AI is, what ML is, how they can really start taking advantage of these paradigms to be able to build a whole new scenario of apps and maybe help make the world a better place with them.
Kirill Eremenko: Fantastic, and how has the uptake been? I’m assuming as you’ve been doing this, so you’ve been a AI advocate for over two years now just in that specific role. Have you seen more and more developers becoming interested in TensorFlow and AI tools?
Laurence Moroney: Yeah, definitely. So, I think it’s a function of the times as well as a function of our efforts. But it’s certainly been one of those things where developers, it’s a difficult job, because very often the landscape is moving so fast that it’s hard for you to keep up. Today you can be a cutting-edge developer, tomorrow you can be a dinosaur, and maybe not that fast, but it’s certainly accelerating. So one of the things that I’ve seen with AI and ML is that a lot of people are identifying that that’s very much the future for me as a developer, it’s something that I have to do.
Laurence Moroney: And so that’s been one of the forces that’s been driving people towards learning AI and learning ML in huge numbers. And another one of the forces of course, has been that it’s been much easier to do it. And with frameworks such as TensorFlow, we’ve tried to make it as easy as possible for a software developer who doesn’t have a Ph.D., someone like me who hasn’t done calculus in 20 years to be able to kind of sit down and start understanding what machine learning models are, how machine learning models work, to start building them, and then to start actually turning those into applications that can be really useful for end users.
Laurence Moroney: So it’s the intersection of trends like that as well as then increased compute power. It can be compute intensive to create a machine learning model because there’s a lot of heavy math going on. But with the dropping prices of chips, particularly GPUs, that extra power now, the GPU in my laptop is far more powerful than a supercomputer of five years ago, for example. With that kind of power being then made available and put into developer’s hands, that all of these trends, I see intersections… intersecting, I should say to make this like a new and really, really cool way to showcase your skills as a developer and hopefully build up careers.
Kirill Eremenko: Love it, love it. So the three trends you mentioned that the third one going from a most recent one, or like in backward order. So computing power is increasing, number two was it’s easier to do AI in ML, we’ll definitely get back to that one, but I wanted to ask you to elaborate a bit more on this first trend you mentioned that developers are starting to feel that this is the future and I need to get in on this. I need to start doing AI and ML. Why is that?
Laurence Moroney: Oh, good question. I think there’s a variety of reasons. One of them is, sometimes there’s always the feeling that every app that can be built has already been built. If I want to sit down and I want to like have a startup and create a new app and, okay, I have a great idea for a to-do list app or something along those lines. I go to the App Store and are hundreds of them. I want to build an app for image processing, I go to the App Store, there’s hundreds of them. So that tends to be that feeling that in order for you to distinguish yourself, you really need to set out with some new scenarios that aren’t previously possible. And having machine learning in an app is one of the things that unlocks that.
Laurence Moroney: Whenever I do a presentation, I always show this slide where it’s, think about activity detection. And if you want to do an activity detection in traditional programming, you have data and you have rules that act on that data. So for example, the data might be the accelerometer on the devices is saying the speed that you’re going at, and you can write if-then rules that say, well, if your speed is this, you’re probably walking, if you speed as this, you’re probably running and if you speed as this, you’ll probably biking.
Laurence Moroney: Those will be very naive rules and because different people run at different speeds and you might run slower uphill than downhill and all those kinds of things. And as a result, using the traditional programming paradigm scenarios like activity detection are so difficult that they become infeasible so we don’t do them. But now with the machine learning, the idea is like well you gather all the data from the devices and lots of people using it, and then they label it. So now you say, “Hey, look, this data is what walking looks like this data is what running looks like, what biking looks like.” And even things like golfing or swimming that you couldn’t really have if then rules for, you can so gather the data and have that.
Laurence Moroney: And it’s no coincidence now that the paradigm has shifted so that those kinds of apps have become commonplace because of machine learning. Things like the Apple Watch and the Google Watch have activity detection on them so that when you’re out running or something like that, it knows that you’re running so that it can calculate the calories that you’re burning when you’re running and those kinds of things. So having that new programming paradigm has cracked open all of these new scenarios that weren’t previously possible.
Laurence Moroney: Those who don’t learn from history are condemned to repeat it, but those who do learn from history are able to take advantage of it. And if I go back to the last big revolution like this that I’ve seen was the emergence of the smart mobile device. And we went from all of our compute being done on desktops and laptops to having compute done on these little black rectangles in our pocket. So having that little black rectangle in our pocket, that’s loaded with sensors made all of these kinds of new scenarios possible.
Laurence Moroney: And the one I always like to joke about is Uber. So before you had mobile devices, if you needed to get a taxi home, if you’re out at night, you’d have to stand on a street corner waving your arms around, or you’d have to call a particular number and then wait on a street corner for them to come and get you and stuff like that. But now you’ve got this little device in your pocket that you can touch a button and then it will summon a car from the internet, that’ll come and pick you up and take you home. That just wasn’t possible on a laptop, you’re not going to carry a laptop with you when you go out to the nightclub and try and call somebody like that.
Laurence Moroney: So that kind of revolution in devices and in different new app types opened up all of these new scenarios of businesses that weren’t previously possible, Instagram is another one. And having a new scenario for app development with machine learning and AI is where I’m excited because it’s a similar type of revolution because it opens up new types of apps that you would just not have been able to build before now.
Kirill Eremenko: Would you say that developers coming into the world of artificial intelligence through tools like Keras and TensorFlow can also, for instance, maybe somebody is not that interested in developing apps, which is definitely, as you pointed out like a great way to go, but do those tools open up new possibilities in the space of just developing AI software or maybe integrating AI in websites for web developers and so on?
Laurence Moroney: Absolutely, yes. So I always like to think about it, this as a new programming paradigm. And whenever you have a new programming paradigm that new programming paradigm can be extended to websites, can be extended to backend applications, can be extended to mobile apps and all of those kinds of things, provided you have the tools. And so in this case, for example, you mentioned websites, one of the things that we’ve been working really hard on is TensorFlow JS, where it’s a JavaScript implementation of TensorFlow so that you can both train models and execute models in the browser using JavaScript. So if you’re a front-end web developer and you want to have ML models that do something fancy within your webpage, you can do so.
Laurence Moroney: Of course, if you’re a backend developer that wants to deploy stuff to the web, then by the fact that TensorFlow JS runs on node, then you’re covered. Alternatively, if you want to build a model that runs inference in the cloud and you want your server to be able to, for example, I don’t know, upload a picture and have a server return the contents of that picture, that’s also possible as a web developer for you to do that. So think about where the inference actually happens. You can have it native on a device, you can have it within a browser, be that on a device or on a desktop, or you can have it running on a server, in which case your browser can talk to it, your desktop can talk it, your mobile device can talk backend.
Kirill Eremenko: Okay, wow, fantastic, love it then. Is TensorFlow JS already available?
Laurence Moroney: Yeah, TensorFlow JS is out there, there’s lots of really fun, cool demos with it. I wish the demos of everything were as fun as the JS demos, but go take the look.
Kirill Eremenko: That’s awesome, we’ll link that to the show notes. I hope you’re enjoying this amazing episode. We’ll get straight back to it after this super quick announcement. DataScienceGo Virtual, have you registered to attend yet? If not, make sure to check it out datasciencego.com/virtual, the dates are coming up June 20th to 21st, it’s a weekend. On the Saturday, we’ve got talks and workshops for newcomers and transitioners, and on the Sunday, we’ve got talks and workshops for practitioners and managers. So whatever level you are, this is the virtual event for you. And it’s absolutely free. Yes, it’s absolutely free, but the numbers of seats is limited, so apply to attend now.
Kirill Eremenko: You can find the event at datasciencego.com/virtual. Come enjoy the talks, have lots of fun, network with your peers. Even if you don’t manage to get in for whatever reason, you will get the recordings afterwards if you register for the event. Once again, the website is datasciencego.com/virtual. No reason not to attend, no reason not to register. So make sure to jump on this opportunity, only a matter of days left until this happens. And I look forward to seeing you there and now let’s jump straight back into this amazing episode.
Kirill Eremenko: And I think we’ve gradually transitioned to the second point. You mentioned a second trend that has become easier. So if you could describe to us, what is TensorFlow? How does it make building AI easier, especially if I don’t have a Ph.D. in computer science?
Laurence Moroney: Well, I don’t have a Ph.D. in computer science, so first of all, and so I always like to think of TensorFlow as an ecosystem as opposed to a product or anything like that because there’s a whole bunch of parts to it. The first and the main part that we tend to kind of use synonymously with TensorFlow is the framework for machine learning. So it’s a set of libraries that are Python based or Swift based that you can use on your developer workstation to create machine learning models. You can either create low-level machine learning models that use something called graph based execution or my personal preference is you can create higher level models that you’ve got a high-level API Keras is our high-level API for that, where you can just define your models in Python code, or like I said, we’re also working on a version with Swift.
Laurence Moroney: So you define your models in that where a model that I’m talking about is, for example, a neural network that has multiple layers, and it’s as simple as saying, this layer has this many neurons, this layer has this many neurons, this layer has this many, or this is a layer type, like a convolution or an LSTM or something along those lines. But it’s one line of code per layer that you define your model, that you train your model, and then once your model is trained, then you can start running inference on it.
Laurence Moroney: So that’s the program or part of it, shall I say, but beyond that, there is a whole ecosystem. There’s JS that I mentioned earlier on, there’s one of my personal favorites, something called TensorFlow Lite. Where what TensorFlow Lite is, think of it as two tools, one, is a converter that optimizes your models to make them mobile-friendly and then the other one is a set of interpreters for those models that run on different mobile devices. So an iOS-based interpreter and Android-based interpreter, and a microcontroller-based interpreter, as well as a Python-based one, that you can then run on like Raspberry PI or other embedded systems that you can execute Python on.
Laurence Moroney: And then beyond that, when you start thinking in a more enterprisey way, where you like large scale infrastructures that you want to run inference on, we have something called TFX, which is TensorFlow Extended, and that’s the big machine learning pipeline that you could use to power things like Google. And so if you want large scale machine learning models in production, that’s what it’s all about. So it’s really that big ecosystem where we’re trying to cover everything.
Kirill Eremenko: Okay, what do I need to know to get started?
Laurence Moroney: What do you need to know to get started? I would say if you have never done any ML before and you really want to start dipping a toe in it, the only thing you really need to know is a little Python. And so the entry-level stuff just, if you understand a little Python, you’re good to go. And I like it in that way, because I think Python is the simplest language in the world to learn. And the ecosystem around it of various libraries makes it really powerful as well as really simple. And so I’d say if you don’t know Python, that would probably be the first thing. You don’t need to be an expert, but just kind of understand the structure of Python and how it works.
Laurence Moroney: And then there’s so many great learning materials out there once a little Python. I created a course on the Google Developers channel called ML Foundations, which is like a 10 video series that just really take you from, okay, I don’t know anything about machine learning, but I want to understand how the whole paradigm works through being able to build a computer vision models and natural language processing models. Basic ones, but it’s enough to kind of get you started to understand how the ball rolls, how the whole thing is working and to kind of just break the ice. And then once you’ve done that, there’s just lots of great materials out there to help you go deeper. I teach some on Coursera, there’s lots of great books. Is now the time to show my favorite book?
Kirill Eremenko: Yes, yes.
Laurence Moroney: This is my favorite book for right now. It’s called Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow by Aurélien Géron, a terrific, terrific book and Aurélien’s a friend of mine, but I’m still plugging it. If we did this podcast six months from now, I’d show a different book, which will be my one that I’m waiting to publish.
Kirill Eremenko: Wow, what’s your book going to be about?
Laurence Moroney: It’s called AI and Machine Learning for Coders. So it’s really about that. I wrote it to really compliment this book because this book is absolutely brilliant, but you have to know a bit about machine learning and AI already to get the most out of it. But if you read my book first, you’ll get a lot more out of this one.
Kirill Eremenko: Okay, fantastic. How was the experience writing a book?
Laurence Moroney: It was good. It was good. I’ve written many books in my times, so-
Kirill Eremenko: Oh yes, you have, that’s right.
Laurence Moroney: I kind of had given up on writing computer books. The last one I wrote was about two years ago on Firebase. And it was just I’d reached my point where I think I’ve written about 20 of them. And then it’s diminishing returns at that point, but there’s a funny story behind why I wrote another book and it was actually to do with this book. And so I get to spend a lot of time in Asia with my work. Well, not right now because of lockdowns and things. But back last October, I was in Tokyo speaking at a conference.
Laurence Moroney: And so I love to visit bookstores, being an author, of course, I love to visit bookstores just in case they might have my book. And this book had just come out so I thought, “Okay, I’m going to go visit some bookstores to see if I can find Aurélien’s book.” And there was another book-
Kirill Eremenko: In Japan?
Laurence Moroney: In Japan, and there was another book I wanted to buy. I’m going to reach back and find it that I’d heard about. And it’s the Manga Guide to Machine Learning.
Kirill Eremenko: I’ve heard of that book. I’ve heard of it. Is it good? I can’t believe you have it.
Laurence Moroney: It’s awesome. I don’t read Japanese, but I love the pictures but it’s all done in Manga to how [crosstalk 00:21:49] machine learning. So I wanted to find this and so there was a bookstore, a famous bookstore in Tokyo that has beehives on the roof.
Kirill Eremenko: Wow, real beehives? Active?
Laurence Moroney: Real beehive with active bees and they make honey.
Kirill Eremenko: Why would you do that in a store? It’s so dangerous.
Laurence Moroney: They’re basically on the roof of the store. So they’re not in the store itself. And then they have a cafe in the store that sells honey from the beehives on the roof. And this is close to the Imperial Palace and all the gardens around that. So the bees pollinate that. And so I’m like, “Okay, I got to visit that bookstore.” And I was looking for this like I said. And I went to visit that bookstore because I wanted to try a coffee with the honey. And I walked into the bookstore and the first thing I saw was this.
Kirill Eremenko: Wow, the Hands-On Aurélien’s, machine learning.
Laurence Moroney: So this machine learning Aurélien’s book. And I had been working with Aurélien on some stuff. So I was kind of texted him, telling him, send him the pictures. “Hey look, your book’s everywhere.” And I went up the escalators to the computer science department and this book was at the top of the escalator. And then I went to look for my Manga book and I found Aurélien’s book everywhere. So after that, I was like, “I really want to write an Aurélien’s book.” So based on the idea of, like I just mentioned, of something that like really where they kind of works complimentary to Aurélien’s book and I loved the idea. So I just finished writing it last week. And looking forward to seeing it-
Kirill Eremenko: Congrats, that’s awesome.
Laurence Moroney: It compliments it so much. It even has a little salamander on the cover of this one, so.
Kirill Eremenko: Nice, very nice. It’s a very needed book. We find that in our audience, there are lots of developers who are transitioning into the space of data science and AI and that’s been the case ever since we started. And I was very impressed by the numbers you shared before the podcast, if you wouldn’t mind saying them again for the community numbers.
Laurence Moroney: Sure, sure. So when I started this journey it kind of started out of, in some ways, frustration with machine learning and with the way machine learning was being taught to developers. And I approached some folks in Google and I was like, “I read a paper and a research paper that showed that there were 300,000 AI practitioners in the world. And then based on various developer surveys is somewhere between 20 and 30 million software developers in the world.” But all of the materials that were out there for learning machine learning and AI were very much catered towards the 300,000, not towards the 30 million.
Laurence Moroney: And I saw that with a great opportunity for us, and particularly, with TensorFlow being such a developer-friendly product that there was a great opportunity for us to kind of double down on making it more developer-friendly so that we could reach the 30 million instead of the 300,000. And at Google, we always like to say, how do you 10X? If you want to do something, how do you 10X? And to me, that was a clear example of 10X. So the 30 million [crosstalk 00:24:53] Well, yeah, definitely. But I was thinking of the 30 million software developers, if you could reach 10% of them and train them to be AI developers, that’s 3 million, now that’s 10X of the 300,000.
Laurence Moroney: And even when you go back and you look at how the 300,000 was measured, and you can always tell a lot about something, not just by the number, but how you measure it. And the only way at the time that it was being measured was by these are people who have their name on a paper. So published around AI and machine learning or something like that, which tells you everything about the emphasis that the industry [crosstalk 00:25:29]-
Kirill Eremenko: Yeah, like research.
Laurence Moroney: Yeah, this was like mid-2017 when I kind of started these conversations and it happens to be a former manager of mine was managing the AI outreach. And I love this guy and we always worked really well. So I approached him and said that and he was like, “Okay, come join us then and see if this is something that we can do.” And then at the time, the plans in TensorFlow were to launch TensorFlow 2.0 with more high level APIs, more Keras based API, for example, to merge Keras into TensorFlow to reach that audience. So it just happens to be one of those beautiful sweet spots and a nice piece of serendipity. So that’s why I came on board to start trying to drive developer awareness of machine learning and AI, as opposed to researcher awareness.
Kirill Eremenko: Got you. Since you mentioned TensorFlow 2.0, could you tell us a bit about the difference? Because it was huge news when it came out. So what’s the difference between TensorFlow 1.x and 2.0?
Laurence Moroney: I mean, there’s many differences, but I’ll just really highlight two of them, I think. One of them is more of an emphasis on high-level APIs like I just mentioned. In the past the APIs were almost researcher focused, they were all about power. Almost it’s like when driving a car that you can have it automatic gearbox or you can drive stick shift. So those people who are performance drivers, drive stick shift. They want to have that absolute control over when the transmission changes gears so that you can have the optimum acceleration curves so that you can accelerate and you can race. The average driver who just wants to get from A to B safely will generally drive an automatic.
Laurence Moroney: And in some ways I like to see that as the paradigm with TensorFlow 1, we were all about power, we were all about having that fine-grain control. And it was all about this kind of graph-based oxocution, what’s oxocution? Execution, this graph based execution so that you have that real fine-grained control over your models and how your models work. Then with TensorFlow 2, we wanted to enhance, not replace, but enhance that with the ability to have high-level API so that you can focus on things like your model architecture prototyping, as well as building out that ecosystem that I spoke about, with TensorFlow Lite, TFX, and TensorFlow JS, and all those other things.
Laurence Moroney: That’d be the first big thing would be really that kind of additional emphasis. And then the second one, which kind of correlates to that was to have eager execution by default. As programmers, eager execution is our default where we see lines of code and this line executes, then this line, then this line, then this line, but non eager based execution or graph based execution is very different where it’s a case of you load this line into a graph, you load this line, you load this line, you load this line, everything’s in the graph and then you execute the graph in one shot. It can be much more powerful, but it’s more difficult to keep up.
Kirill Eremenko: It’s kind of difference between interpret and to compile languages, is that correct?
Laurence Moroney: Kind of, kind of, I mean, even when something is compiled, it still executes line by line, [crosstalk 00:28:42] compilation. I mean, yeah, it is kind of like that though, but the whole idea was that the graph based is faster it’s more optimized, it’s more powerful, but it’s not quite as friendly for the program or for the developer as eager based. So having eager based having high-level APIs, like Keras has become first-class citizens, as well as like building out the full ecosystem, like with TensorFlow Lite, JS, TFX. That was really the emphasis around TensorFlow 2.0. I think that’s why the excitement kind of took off because going back to my 300,000 versus 30 million analogy, this is certainly much more on the sweet spot of the 30 million than it had been. And that’s why we’re continuing to invest in that.
Kirill Eremenko: Fantastic, I also heard there’s quite a bit of people complaining or unhappy that’s when you switched from TensorFlow 1.x to 2.0 you couldn’t transfer your AI models; they weren’t compatible with the new version. Is that going to be the case going forward or are they going to be compatible from now on?
Laurence Moroney: I think the goal is always to make them compatible. But, of course, whenever you create something new-
Kirill Eremenko: Especially that’s radically different.
Laurence Moroney: Yeah, and you can always have regression problems whenever you create something new. So as you continue, any products, not just TensorFlow, you release a new version of iOS, you got to make sure your applications continue to work with the new version that you don’t end up with regression issues. I think, in this case, there was the paradigm change rather than a breaking change is the biggest kind of leap that one has to make. Where if I’ve spent two years building graph based models, now, there’s this new option of creating these eager based high-level Keras models. Do I continue working in graph-based models or do I make the change? I think is one of the things that people were probably a little bit confused about.
Laurence Moroney: We do want to make, and I do want to underline that graph based models are still a first-class citizen. It’s just by the fact that we brought in eager, we brought in high-level APIs for making Keras-based models as part of TensorFlow doesn’t mean that the graph-based ones are going away. It’s just the idea is to open up more scenarios as opposed to continuing to double down on a single scenario.
Kirill Eremenko: Got you, understood. Let’s shift gears a bit and talk about careers. So I loved your example that you gave just before we started the podcast of bytecode and for instance, like a Java developer. Do you mind sharing that again? Because I think it lays a great foundation about what it is to use AI in business applications and in different types of applications these days.
Laurence Moroney: Sure, so I always like to think about it this way, that there’s a different task involved in learning machine learning as there is into learning programming models. And so, my focus is generally on programming models. Like I mentioned, my book is called AI and Machine Learning for Coders. With programming models, then you’re writing code, you’re creating layers in the neural network, you’re defining how that neural network is going to compile with a loss function and an optimizer, and those kinds of things. You don’t necessarily need to know all of the underlying mechanics of how these things work, and particularly all the underlying math of how these things work in order to be successful.
Laurence Moroney: And the paradigm I always like to use is, if you’re a Java programmer, if you understand how a compiler turns your code into bytecode and how that bytecode executes, that might make you a better programmer. But you don’t need to know that in order to be a programmer. And so, I always like to say it in the same way, so if you want to get into machine learning, often when you start reading a lot of the literature and even my favorite manga book kind of does it, it tends to then start going into a lot of the math.
Laurence Moroney: There’s a process for machine learning called gradient descent, which is very, very heavy on calculus. And often, when you’re learning machine learning it’s the first thing that you learn because it’s the thing that underpins most machine learning. But if you feel that you have to learn the calculus to understand how gradient descent works and how that empowers machine learning, the paradigm for that is starting with looking at bytecode before you start looking at Java code, or starting looking at MSIL before you look at C#, or starting to look at assembly or ones and zeros before you look at high-level coding languages. So the thing that I always want to encourage the 30 million was, you don’t really need to do this stuff in order to be able to do machine learning.
Kirill Eremenko: Absolutely, the example I use very similar to yours is, driving a car to get from A to B, you don’t need to know what are the camshaft versus a crankshaft and on to the hood. You don’t need to know that stuff. And that’s what we teach. I love that you say that because often I face criticism from people I talk to, especially in academia. They’re like, “Well, you got to teach the math.” But in our courses, we don’t teach the math. We focus on the practical plus the intuition. And by intuition, what I mean is, we explain intuitively how does gradient descent work? You know that example of the shape and the ball falling down and they’re fighting the local minimum and so on?
Laurence Moroney: And you’re exactly right. And I would like to add, first of all, if you know how the ball gets to the bottom, that can make you a better machine learning developer, but you don’t need to know that in order to start. And if you know how your car runs, and how the camshaft and all that kind of stuff works, that can make you a better driver and look after your car in a better way, but you don’t need to know that in order to drive.
Kirill Eremenko: And most drivers don’t know that, 99% of drivers drive all the time not knowing that. They need to know how to press gas, need to pass the driving exam, where to put the petrol in, and they can get to A to B. You can live your whole life and never… I hate to admit this, but I often don’t even know how to put oil into my car. I have to ask somebody to help me out with that, and I’m fine driving.
Laurence Moroney: I think the selling point for me to buy an electric car was, they told me, I don’t need to put oil in it.
Kirill Eremenko: That’s great, awesome. So what can people do to kind of overcome this fear? Because it’s natural to have this kind of, “Ooh, AI, math.” Even knowing that you don’t have to learn math, you still will be like, “How do I get over this fear to see that I really don’t need to know the math unless I really want to?”
Laurence Moroney: Yeah, I think just to go through some of the learning scenarios. And so, earlier I mentioned, I have a 10-part course called ML Foundations, which is on the Google Developers Channel.
Kirill Eremenko: This is on your YouTube?
Laurence Moroney: Yeah, it’s on YouTube, Google Developers channel. And my goal with that was, it’s 10 videos, each is like six or seven minutes long. No, sorry, they’re a bit long, they’re about 10, 15 minutes long, sorry. And those 10 videos and the idea is with each one of them, if you go through them, I try to give you a very simple coding scenario. Like the very first video was I kind of teach what machine learning is, what the scenario is, how it kind of is all about pattern matching ultimately, and then show some simple codes for inferring the relationship between two numbers. And so, the people can see that it’s all about inference. And then by the time they get out to episode two, they’re using exactly the same paradigm and exactly the same pattern to do a basic computer vision scenario. By episode three, they’re in an advanced computer visions scenario, stuff like that.
Laurence Moroney: And I think one of the things is, from working with students, I’ve found that when people kind of start doing something and seeing that it works, and they do it themselves and they see that it works. They’re not downloading a thousand lines of code written by somebody else that they have no idea what it does to get it done or working, but they’re writing 10 lines of code, then 20 lines of code, then 50 lines of code and it actually works, that builds up that internal confidence, gets rid of the imposter syndrome, and makes people realize you know what? I think I can do this and I think I’m beginning to get it.
Laurence Moroney: And that to me is the key to helping developers. Because being a developer, I think, is the greatest career in the world, but it does have its challenges. And one of the challenges is that there’s so many frameworks coming at you thick and fast, from so many vendors, and so many paradigms. You don’t have time to learn them all, but if you have the ability to be successful with one quickly, that’s the kind of thing that can really help you kind of gain an affinity with that framework and maybe gain an affinity with that career direction.
Kirill Eremenko: And that’s what I love about Keras and TensorFlow, that you only need five, 10, I don’t know, 20 lines of code to already have a neural network set up and running. You don’t need to be writing a thousand lines of code. It’s kind of almost like drag and drop, building something out of legos. Put this here, put this here, specify, do your hyperparameters and boom, it’s working.
Laurence Moroney: Yeah, exactly. I mean, you’re not going to build a world-changing app with that but the idea is that… What I really love about it is that the paradigm of like it’s having a machine infer the roles that match the data to the labels. Once you know machine learning, you’ll know exactly what I mean when I say that. But having that paradigm and seeing how that paradigm works from the smallest hello world application all the way up through the most advanced applications. And then you can implement that pattern, that paradigm in five lines of code for a simple scenario and then take that through to the more advanced scenarios as this. To me, that was really eye-opening and one of the great differences about machine learning. And one of the things that makes getting started quite simple and being able to move to advanced scenario is pretty easy.
Kirill Eremenko: Fantastic. I find two things probably are quite important for somebody jumping into a new area, because that’s what we do. We help people transition to data science, machine learning, and things like that. And comments that we hear from people and just over the years, we’ve discovered that two very important things are, having a community that you can ask questions and get support and answers. You’re not alone learning the stuff, that you don’t have to sit and read through a book every time. You can just ask a question, or somebody has already asked the question and somebody has answered it.
Kirill Eremenko: And the second thing would be, that you have a path, you know where you’re going, the direction. And I know you mentioned before we started that you have TensorFlow certificate exam and it’s quite a tough exam, but at least it’s like a point towards which you’re going. Do you mind elaborating on those two, the community part, and the pathway of where a person needs to be going, and what the certificate exam is that you have?
Laurence Moroney: Sure, I’ll start with the second part of that, the certificate exam, and all that. So this was born out of, because it’s such a new thing, machine learning and AI, there’s a lot of fundamental misunderstanding about the skills required to get a job in it. And I have seen a lot of things where people will be like, “Here’s the question I got in an interview for a job. And it was something along the lines of, explain how the Adam optimizer works.” And it’s similar to what we’ve talked about already around, you don’t need to know how the engine of your car works in order to be able to drive.
Laurence Moroney: But there was so much misunderstanding in the field that it was like people who are interviewing don’t know what to ask. So they were like, “Okay, well, what’s gradient descent? How does an Adam optimizer work?” Because that’s how the skills were actually taught. The very mathematical concept of machine learning, it’s very valuable skills, I don’t want to dismiss them but when somebody was looking for a job, it was like, “I know how to program computer vision. You’re looking for something in computer vision. I can’t go to a whiteboard and explain the math behind how an Adam optimizer works and should I need to in order to be able to get the job.”
Laurence Moroney: So that was part of this got born out of things like that and it was evidence that we’ve seen in the industry of those kinds of things happening. So we did some research and we kind of dug into, well, what are the skills that are really needed right now by developers if they want to become involved in companies that create AI apps or create ML apps? And there were really three. And in order of sequence, it was number one was definitely computer vision. So most people that we saw that we’re hiring in ML skills, were looking for computer vision skills. How to build CNNs, to be able to understand computer vision, to build image classifiers, object detectives, those kinds of things.
Laurence Moroney: Number two was natural language processing. Somewhat behind computer vision, but it’s still number two, is natural language processing. And again, to be able to build models that handled NLP. And then, number three was sequence modeling. So being able to understand sequences of data, to be able to predict the next thing, like what’s the weather going to be like tomorrow, who’s going to win the world series or whatever, those kinds of things. And so, I came up with the strategy then of creating courses, and that’s the TensorFlow in Practice Specialization that’s on Coursera, as well as a number of courses that we’ve had universities create, that teach those fundamental skills.
Laurence Moroney: And then, once the world had been seeded with lessons, teaching those fundamental skills, then it’s a case of now, if we launch a certificate exam that tests those fundamental skills, and when somebody passes that exam, we have a way of showcasing those folks. So it’s on tensorflow.org/certificate. And so, you pass the exam, you get your name and your badge on there that we show, hey, this person has passed this exam. They have the skills; computer vision, natural language processing, sequence modeling, basic TensorFlow, those kinds of things.
Laurence Moroney: So now, if a company is looking for ML skills, instead of them looking up academic research papers and saying, “Oh, maybe I should check if this person can explain what an Adam optimizer is,” that they can actually say, “As a developer, can you build a CNN? Show me your code. Oh, here’s your certificate that Google shows.” And to kind of really just try to seed the whole employment in that way, or to help seed the whole employment in that way so that we have a way of showing that these people have these skills and these are the skills that we found that companies are looking for so that we can bring the two together.
Kirill Eremenko: Wow, love it. I’m just so excited listening to this. This is amazing. And you said this is recent, a couple of months?
Laurence Moroney: We launched the certificate exam at the TensorFlow Dev Summit back in March, and so it’s like about-
Kirill Eremenko: March 2020?
Laurence Moroney: Yeah, March 2020. So we’ve launched it then. And so, we built a site, we’re seeing more and more people passing every day and it’s [crosstalk 00:43:57] site-
Kirill Eremenko: How many have passed so far?
Laurence Moroney: I don’t have the numbers on hands. It’s in the hundreds.
Kirill Eremenko: In the hundreds, wow. If you’re listening to this podcast, you can be one of the first 1000 to pass this exam. That’s amazing.
Laurence Moroney: It was funny because we have a map on the website that you can go to the map and where people are from, does it a ping in the map? So there’s a lot of times like, are you going to be the first in your country to do it? I grew up between Ireland and Wales and the last time I visited, there was a ping in Ireland so I was so excited. But there wasn’t a ping in Wales yet. So maybe-
Kirill Eremenko: You should pass it to put up a-
Laurence Moroney: I wrote the exam, so I’m not allowed to take it.
Kirill Eremenko: Amazing, wow, love it. That’s that’s really exciting news. If you want to add a lot of structure because you’re right, I sometimes feel sad for hiring managers who are tasked to build a data science team and they don’t know what data science is and they put everything under the sun into the job description, it’s crazy.
Laurence Moroney: Yeah, it really is but, I mean, to be fair, I think that’s just because the demand is so high but the maturity of the industry right now is still relatively low that people really don’t know what to do there but as the industry matures and that’s one of the things that we’re trying to do is to help it along in maturing then I think those kinds of issues will go away. I’m just thinking if you remember a few years ago when mobile development was sky-high red hot, the iPhone came out, what was it? 2007. And you would see job listings in 2010 looking for somebody with five years [crosstalk 00:45:38], that was the same kind of thing.
Kirill Eremenko: Yeah, yeah, yeah, exactly. But I was going to say that now is the perfect time. Once this industry matures, it’ll be harder and harder to breakthrough. Like accounting, it’s a great industry, I studied accounting. But you got to have to climb the career ladder, you have to progress. There’s very few cases I can even think of that somebody studies accounting or just learns it online and then boom, they’re the top accountant in the world, no, it’s a very mature industry. Well, same thing’s going to happen with AI, maybe 10, 20, 30 years from now. Now is the time if you’re a developer and you’re thinking of getting into AI and ML, now’s the time because it’s so immature, you can skyrocket your career. You can change the world because it’s so new and everything’s all over the place.
Laurence Moroney: Absolutely, and I always like to encourage people who are underrepresented in our industry, women are underrepresented, many minorities are underrepresented in our industry that I particularly encourage them that now is the time because I think when you have these skills that people really, really, really desperately need, they tend to see through that prejudice, it’s one of the things that removes those prejudice lenses.
Laurence Moroney: And so I graduated in 1991 and that was like the worst recession in history at that time in the UK. There’ve been worse ones since but at that time it was the worst one. And you probably can hear it a little bit in my voice, although it’s been a long time but I’m actually Irish. At that time, Irish people were very, very heavily discriminated against for a variety of reasons.
Laurence Moroney: And I remember walking out into the world with my freshly minted degree, thinking everybody would give me a job and nobody would give me a job because of the recession. And then in some cases, even when I went to some interviews or to some job fairs, people wouldn’t talk to me when they heard my accent, it was crazy. But at that time, 1991 into 1992, this new technology was beginning to emerge. That seemed like science fiction, it was called the web.
Laurence Moroney: Nowadays we all use the web and the web is part of our everyday lives but it was absolutely cutting edge, bleeding edge in the early 1990s. And I realized that when I skilled myself up in the web, I went back and I studied like a post grad, I graduated from that in ’93. I skilled myself up in the web then suddenly employers were lining up to talk to me. The folks that wouldn’t even let me into the room because they heard my accent were now lining up to talk to me. And ever since I’ve been able to build a wonderful career as a result. So I think as you were saying, it’s like you could get into AI and ML now and it’s the opportunity for you to rocket boost your career. But it is also, for folks who are finding it hard to get into traditional programming careers or who are sidelined in their traditional programming careers, it’s a great way for you to kind of bust that glass ceiling too.
Kirill Eremenko: Well, thank you, that’s really insightful advice and I think it brings us nicely to the topic of community. What is the TensorFlow community like these days?
Laurence Moroney: It’s vibrant, it’s pretty amazing. So we have a community manager, you should have her onto the talk someday, she’s great Joana. But the community, it’s just growing amazingly. Last week, I had to do a webcast to the TensorFlow Community in Korea because they had just hit, I think it was 75,000 users. It’s growing and that’s just in one country. So I think the community is growing. And I think part of it has been that as more and more developers are getting involved and it’s becoming more and more welcoming for people who don’t have Ph.D.s or who don’t have that kind of deep level of math. So that we’re all kind of in this boat together where we can all help each other out and that realization is kind of really dawning heavily, I see communities just growing hugely. So it’s really exciting.
Laurence Moroney: I teach a course on Coursera, a number of courses on Coursera but one of my courses there just crossed the 150,000 students. That just amazes me, that level of engagement. And the YouTube channel for TensorFlow, I run that and we just crossed 250,000 subscribers last week. I think TensorFlow itself was it, I think it was 10 million downloads. Things like that we just see that the community is vibrant and it’s a lot of fun to be a part of it.
Kirill Eremenko: Fantastic, by the way for everybody listening, I highly recommend subscribing to the YouTube channel, there’s really cool videos that are coming out, you can see Laurence presenting there.
Laurence Moroney: Way too much of me, that’s the only problem.
Kirill Eremenko: So is there a centralized place where people can ask questions, get answers, where does the community hang out mostly?
Laurence Moroney: Yeah, so on tensorflow.org, there’s a community page which lists details of all the in-person community you can join in your local area, as well as a number of distribution lists of email that you can ask questions. The first and the granddaddy of them all is discuss@tensorflow.org, you can send emails there. But yeah, if you go to the tensorflow.org/community site, you’ll see details on all of the communities, I definitely would encourage joining an in-person community. I said, if you can, I mean, obviously right now, we’re still in COVID and in-person stuff is more difficult. People are doing them virtually on Zoom and Google Hangouts but I think if you can join an in-person community, that would certainly be a great way to get connected.
Kirill Eremenko: Or start an in-person community if there isn’t one in your city.
Laurence Moroney: That’s a great point, absolutely. And details on how to do that are on that site. They call them TFUG, TensorFlow User Group, kind of spinning up all the time and there’s details on how to do that on the tensorflow.org/community site.
Kirill Eremenko: Okay, fantastic. AutoML, what are your thoughts on AutoML and from being in the industry and being an AI advocate, do you think AutoML will replace data scientists in the long run?
Laurence Moroney: No, no, I mean, like anything else it’s a tool to help us be more efficient. So one of the things that you find when you’re building models is hyper-parameter tuning, what loss function do I use? How many nodes in this layer, what size filter do I use on my CNN? All of those kinds of things that it’s a lot of trial and error unless you are a deep expert on this stuff, who knows the papers backwards, that’s not me. There’s a lot of trial and error there and really in optimizing your model and AutoML is one of the things to really help you do that.
Laurence Moroney: And the same way as does a debugger make a programmer obsolete, does profiling tools make a programmer obsolete? It’s the same kind of thing like that, it’s just something that you can use to make yourself a more efficient developer and that you’re always going to have the need for the person to get the ball rolling, to define the architecture and then to use something like AutoML as a tool for that.
Laurence Moroney: In addition to AutoML, I would also encourage because AutoML is this big infrastructure for creating large scale models but I’d also kind of encourage that there’s something called Keras Tuning which is an open-source API that you can kind of sort of do the same thing, you’ll create your model, you’ll think about the loss functions optimizes model architecture, all that kind of stuff that you’re going to do. But then with Keras Tuning, what you can do is then say, “Okay, try this range of this hyper-parameter, try this range of neurons, try these ranges of filters.” Those kinds of things. And what it will do is it will basically go through every option, train your model with every option, you have a target thing, for example, model accuracy or loss. And then it will report back to you when it’s done of the model architecture that gave you the desired loss or model accuracy. It’s kind of AutoML in that way.
Laurence Moroney: And I actually like to do that a lot, I have a GPU under my desk over here behind me and sometimes I kind of sort tweak for a few hours to come up with what I think is the best model and then use Keras Tuning on that, run it overnight and see what Keras Tuning came back to me if nothing else just to validate that I had the model right but often it’s a case of tweak this or tweak that and you’ll get a better model. So not all of us can have an AutoML infrastructure at our fingertips that we can tune like that but with Keras Tuning open source and free you can sort of do that stuff. And you’ll see, once you start doing that it’s making you a more effective developer as opposed to replacing you as a developer.
Kirill Eremenko: Yeah, love it, love that. Okay, well and to finish off, what does the future look like? What do you see in the future of AI, TensorFlow and developers intersecting with AI developers, let’s say three to five years from now?
Laurence Moroney: Sure, so instead of what the future looks like, I always like to say, this is what I think success looks like. And what I think success looks like is that we have a future where this isn’t a special cornered scale, that this is just a part of every developer’s toolbox. Once upon a time mobile developers with this special breed, there were the general developers and then there were the mobile developers but now everybody’s a mobile developer as well, almost everybody has to have mobile skills as part of their development toolbox or web skills or database skills as part of their development toolbox.
Laurence Moroney: And I think what success will look like is that when AI and ML skills aren’t these specialized corner case but they are the thing that’s part of every developer’s toolbox and it becomes normalized in that way. And then once we have that and the ML paradigm, and building things with the ML paradigm becomes a normal way of building applications, then what I was talking right back at the beginning was all these new scenarios that none of us have even thought about yet, we’ll have applications for those scenarios and we’ll have these new ecosystems around those applications. That’s to me is what success would look like. And hopefully, it won’t take as much as long as five years hopefully, we’ll be there sooner.
Kirill Eremenko: Fantastic, never know with these exponential trends, might happen sooner for sure. Fantastic, well Laurence, thank you so much, it’s been a huge pleasure. How can our listeners find you? Maybe follow your career and things that you’re doing in the world of TensorFlow?
Laurence Moroney: Sure, so youtube.com/tensorflow is pretty good. And I make a lot of videos on there, I run that site. And it’d be nice to get more subscribers, so that would be good. To find me personally, Twitter is probably the best, LMaroney, L-M-O-R-O-N-E-Y, so on there. Otherwise, you can just reach out to me, like LinkedIn, Twitter, any of those. And I try to answer as many questions as I can.
Kirill Eremenko: Fantastic, Laurence thank you so much it has being a huge pleasure having you on the show.
Laurence Moroney: Thank you, it’s been a blast.
Kirill Eremenko: Thank you everybody for being part of today’s conversation, it was super amazing knowing that you will be listening to this because the insights that Laurence shared were truly incredible. I enriched my knowledge of TensorFlow and the TensorFlow ecosystem vastly and for sure, the advice about getting into this space and making part of your career if you’re a developer or, for that matter, if you’re just looking into the space of artificial intelligence, just taking the leap and getting into this world of TensorFlow and how you can do that, I found that advice absolutely invaluable.
Kirill Eremenko: My favorite part of the podcast was when Laurence and I were discussing the current state and the future of the world of data science and machine learning and AI and that’s right now, it’s not really mature, it will become mature but right now is the time to get into the space. I felt super pumped and excited at that moment and I hope you did too. And if you’re sitting on the fence and thinking, whether it’s worth to take the leap or not, this is the time to make that call and give TensorFlow and artificial intelligence a try.
Kirill Eremenko: And as usual, you can get all the show notes for this episode at www.superdatascience.com/373, that www.superdatascience.com/373. There you’ll find any links we mentioned, any books, any URLs such as Laurence’s LinkedIn, the YouTube channel, the tensorflow.org page, and anywhere else where you can get more information so check that out.
Kirill Eremenko: And if you enjoyed this episode and somebody who’s a developer or somebody who’s just been meaning to get into the space of artificial intelligence and explore it but maybe they’ve had some fear, maybe they haven’t had the determination or courage to take that first step into the space of AI, send them this episode, help them advance their career, help them overcome that fear. Very easy to share, just send them the link, www.superdatascience.com/373, and you might actually impact someone’s career for the better and help them master artificial intelligence. How exciting is that? Thank you very much once again for being here today, a huge thank you to Laurence for coming on the show and sharing the insights. And I look forward to seeing you back here next time until then, happy analyzing.
