SDS 102: Intuition vs Mathematics in Data Science

Hello and welcome back everybody to SuperDataScience podcast, what I wanted to share with you today is intuition in data science. This is not like intuition as in how to make predictions based on being intuitive or using your gut feel in data science. There definitely might be some space for that and maybe we can discuss that another time.

What I mean by intuition in data science is the intuitive understanding of concepts in data science. And so, if you’ve taken any of our courses you will notice that very often I include intuition lectures. For instance, even courses like on AI, Deep Learning, Computer Vision, Machine Learning, the way Hadelin and I structure our approach is he takes care of the coding part and the application, the real-life applications and I take care of the intuition side of things, on the intuitive understanding.

The question is why intuition? Why is intuition important in data science? The reason here is that a lot of the concepts in data science are actually very technical, are actually driven by very complex mathematics and statistics and when you look deep into them, behind the hood of these libraries in machine learning or deep learning or even more basic things like logistic regression which is a standard classification algorithm in data science and it’s been used for decades in statistics, if you look behind them, the math is very complex and therefore it looks like there’s a huge hurdle to get over in order to become a data scientist.

But I really don’t believe that. I really don’t believe that it must be so complex and I’ll explain why. This is exactly what this podcast is about. That data science doesn’t need to be that complex. The way to think about it is my favorite analogy is a car. We all know that cars are quite complex inside. There’s chassis and there’s shafts and there’s wheels and engines and pistons and lots and lots of stuff, like lots of little components and big components inside a car. Personally, I’ve never pulled a car apart in my life. I’ve looked inside, I’ve changed the oil a few times but so far anyway.

[Laughter] We were on a road trip over at Atlanta and we entered this car and we didn’t know how change the oil, we had to- actually it was so embarrassing- we had to ask the guy at the petrol station to help us out, how to change the oil. But even to that extent, I know now how to change the oil I’ve maybe done it a couple of times, but don’t even ask me about what is inside and how the torque goes from the engine to the back wheels or front wheels. I could probably look that up and explain but right now I don’t know it.

Unless you are an auto mechanic, you’ve also probably never pulled a car apart. You probably know a couple of things here and there but every single little detail of what’s inside a car what’s inside the engine even now with more complex things like the electronics behind a car, it’s very unlikely that you know them. Yet, all of us can get inside a car and we can drive it. We can get from point A to point B, we can go to the shop, we can pick up the kids, we can go for a holiday, whatever we want, we can drive the car. Even without knowing what’s inside it.

That’s exactly the same thing that I see for data science. You don’t need to know all the mathematics behind it. You don’t need to know what exactly is going on, how exactly this algorithm works, to the nitty gritty, to the nuts and bolts, what formula is used here, what formula is used there, and so on. All you need to know is the intuition behind it. You need to know what’s the intuit, how to understand intuitively what’s going on when you’re applying a K-means clustering algorithm or what’s going on where you’re running a Apriori algorithm or how an artificial neural network works, or how a convolutional neural network works. Lots and lots of different things that we apply in data science. You just need to know the intuition behind them without having to delve into the deep mathematics and statistics, and then you can also apply them.

Because the bottom line for a data scientist, a successful data scientist is not somebody who knows all the math behind all the algorithm that he uses. A successful data scientist is somebody who knows what they do and why they do it, those algorithms, and how to use those algorithms to apply to real worlds situations to derive value. That’s what businesses want. A lot of people make the mistake thinking that to be a successful data scientist I have to learn all the math. But you need to think about what is actually required of a data scientist.

If you want to become a data science researcher, and you want to develop these algorithms and push the envelope in the space of data science in the world and make it even better and more powerful, then yes. But if you just want to help businesses help charities apply data science in your own projects, then you just need to know overall how these algorithms work and understand when to use which one and what exactly they do, and you need to know how to apply them in real life and derive value. That’s it, just as with a car. In order to bring value with a car, you just need to know how to drive it, you don’t need to know how to pull it apart.

So, that’s something to think about and hopefully, that’s a bit of encouragement for you if you’re thinking that- like maybe if you’re studying data science or you’re starting to get into data science and you think that the world of machine learning is complex or the world of artificial intelligence or deep learning is complex. It’s as complex as you want it to be. It’s as complex as you want to get into it, but it doesn’t have to be complex in order for you to build a successful career and derive lots of value. You can go your whole life without knowing all the formulas behind an algorithm and use it as a virtuoso and bring so much value and create so many great things for businesses or for the world.

There you go, that’s intuition vs mathematics in data science. Hope you enjoyed this episode. Share it around with your friends. If you know somebody or colleagues, somebody who’s maybe a bit apprehensive of getting to the space of data science because they think it’s too complex, maybe this will help them get started.

I look forward to seeing you here next time, and until then, happy analyzing.