SDS 803: How to Thrive in Your (Data Science) Career, with Daliana Liu

This is episode number 803 with Daliana Liu, host of the Data Scientist Show. Today’s episode is brought to you by AWS Cloud Computing Services, by Babbel, the Science-backed language learning platform, and by Gurobi, the Decision Intelligence leader. 

00:00:20

Welcome to the Super Data Science podcast, the most listened to podcast in the data science industry. Each week, we bring you inspiring people and ideas to help you build a successful career in data science. I’m your host, Jon Krohn. Thanks for joining me today. And now, let’s make the complex simple. 

00:00:51

Welcome back to the Super Data Science podcast. Today we’ve got the data science rock star Daliana Liu on the show. If you haven’t already heard of her, Daliana was formerly a senior data scientist for Amazon Web Services, where she spent seven years analyzing hundreds of experiments to support product launches. She also led data science training for Amazon engineers and built ML models for customers in the sports industry. She’s renowned for her content creation on data science careers, particularly career growth strategies, allowing her to accumulate over 280,000 followers on LinkedIn, and develop a hardcore following on her popular podcast, the Data Scientist Show. 

00:01:27

Today, she specializes in one-to-one career coaching, as well as coaching groups through structured programs like her upcoming Survive and Thrive in Data Science and AI Careers course. Today’s episode is well suited to anyone who’d like to thrive better than ever professionally. It will particularly appeal to data scientists and related professionals, like data analysts, ML engineers, software developers, and so on, but most of the advice Daliana covers is beneficial to anyone.

00:01:52

In today’s episode, Daliana details common, unhelpful career mindsets and how to overcome them, how to find the role you really want as opposed to the one you think you want, how to find your niche in a fast-moving field, how to offset common professional issues like imposter syndrome, distraction, and burnout, her top tips for accelerating a technical career, and the must-know tech skills for data scientists in today’s market. Are you ready for this fun and informative episode? Let’s go. 

00:02:24

Daliana, a rock star in our midst. Welcome to the Super Data Science podcast. Where are you calling in from today, Daliana? 

Daliana Liu: 00:02:30

At this moment, I’m in Austin, Texas. 

Jon Krohn: 00:02:34

Nice. And you might be visiting Ken Jee in Austin, is that right? 

Daliana Liu: 00:02:39

Yeah, I saw Ken Jee last month and I visited his studio. It was really cool. 

Jon Krohn: 00:02:47

He’s got a studio? 

Daliana Liu: 00:02:47

He has a studio at home, yeah. 

Jon Krohn: 00:02:50

Wow. Oh, that’s cool. And so, Ken was in episode… If people don’t know Ken, he’s huge on YouTube, although also recently announced that he is not doing data science content creation as much or at all anymore, so that’s what he kind of… He built his hundreds of thousands of YouTube subscribers on the basis of his data science expertise. But I guess I watched a video recently that he published, that he’s moving into sports. 

Daliana Liu: 00:03:20

Yeah, sports analytics. That’s something he always have been working on. 

Jon Krohn: 00:03:24

Sports analytics, yeah. 

Daliana Liu: 00:03:25

Yeah, I think, as a content creator, every couple years everyone has this pivot moment. And yeah, we’ll get to it later, I think I just went through a moment when I realized I need to pivot. 

Jon Krohn: 00:03:39

Yes. In fact, that’s pretty much where we’re starting. I just wanted to quickly, before we do that, say thanks to Ken or Harpreet Sahota, we’re not sure. Daliana and I can’t tell, we can’t remember for sure, we’ve been connected for years, following each other for years, filming this episode has been in the works for a long time. Maybe I’ll be on your Data Scientist Show, which we’ll talk about later in the show- 

Daliana Liu: 00:04:00

Definitely. 

Jon Krohn: 00:04:01

… as well in this podcast episode. So yeah, Harpreet, he was on this podcast most recently, in episode number 693. Ken Jee was in episode number 555. Both of them outstanding content creators and amazing connectors. Just so friendly, always happy to make additional connections. And so, between one of those two people, I guess, some years ago, you and I were connected.

Daliana Liu: 00:04:26

Yeah. 

Jon Krohn: 00:04:27

So yeah, let’s talk about that transition. You were, until recently, a full-time data scientist. You were a senior data scientist at a fast-growing AI startup called Predibase. And prior to that, you were at a little known company called Amazon Web Services. And you have publicly stated, and so I hope it’s all right that I’m mentioning this on air, that you were burnt out doing that on top of all the content creation you’re doing, so you quit the full-time data scientist role, you pivoted to being full-time focused on career coaching and your podcast. 

Daliana Liu: 00:05:04

Yeah. 

Jon Krohn: 00:05:05

So yeah, tell us about that transition. 

Daliana Liu: 00:05:06

Yeah, so the whole content creation journey started around 2020. I think around that time, there are more data scientists roles and people are very confused about what do data scientists do. And people think, as a data scientist, you do AB testing, machine learning, everything, deep learning. And I think there are not a lot of people talking about what do we actually do? What are the frustrations we have as data science, machine learning practitioners? 

00:05:40

So I started content creation on LinkedIn. And around that time, I also got promoted to a senior data scientist at AWS, building machine learning solutions for a lot of customers in the sports industry, so it was very exciting. You get to learn different type of business and different type of models. And since 2020, I’ve always created content on LinkedIn and I started my own podcast, the Data Scientist Show. So, for over three years, it felt like I had two jobs. I had my day job, I’m a data scientist, and then I need to think about writing, editing, coming out with new ideas and later trying to keep up with a weekly publishing schedule. Definitely feel like a grind.

00:06:38

And last year, around September, I left my full-time job as a data scientist, because the podcast was growing and was doing really well, and I also started to collaborate with other data and AI companies to co-create educational content. And I was doing that for about a year, and you can say my role switched from a practitioner to a podcaster, content creator, and my customers are the business and I’m an educator. So maybe I didn’t talk about a lot on LinkedIn, I also started a career coaching course on Maven. So it was a live course, but because it was the first time I launched this course, I only promoted once. I had about 10 people joined and I want to really give people the experience. 

00:07:43

And then I realized, although I went over time every time and I didn’t make a lot of money compared to the business I built around creating educational content, doing partnerships with other tech companies, but I felt very fulfilled. I really enjoy giving people advice. And what I noticed, sometime when I’m on a Uber, a lot of Uber driver, they’re going through career transitions and then they’re scared of trying something. I just feel I want to give them advice. And then, that lead to the feeling, “Oh, maybe I need to take a break to figure out what I really want to do.” Because doing career coaching and doing weekly content and also working on those partnership conversations, a lot of negotiation, you need to think about delivering results, meeting their community, their marketing goals. So I think that’s the moment I realized, maybe I was a little burned out. 

Jon Krohn: 00:08:55

Yeah, that does sound like a lot. And so, those Maven courses, they’re live training, they’re not recorded sessions. And we’re actually going to talk about… I don’t know if you have multiple courses or just the one Advance Your Data Science Career with Proven Strategies. Is that the one, or do you have multiple of those? 

Daliana Liu: 00:09:13

It’s a one course. So, last year I called it Advance Your Data Science Career. It’s the same course, but now I added more content. I call it How to Survive and Thrive in Data Science in the Era of AI, because the data science landscape has changed. Although I stopped my podcast, but I had a lot of off the record conversation with practitioners. And I know, sometimes some job requirement is different, while some functions still remain the same. And also, I realize a lot of data scientists are scared about layoffs, so they also want to know how to build their personal brand in the industry. 

00:10:02

So besides the soft skill, how to sell your project to stakeholders, how to manage your ML projects, how to give commune presentations. We’ll also talk about the updated industry reality and how people learn skills about personal branding, how to build influence. Maybe some people also want to teach a course, or maybe some people want to build their independent consulting service. 

Jon Krohn: 00:10:36

Nice. Yeah, it makes a lot of sense. On the note of your podcast and how you’re not creating episodes as much these days, we actually had an audience question. So I posted, as I do with some of our guests as they’re coming up, I posted a week before recording this with you that you’d be on the show and does anyone have any questions? And we had Teslim who, he lives in the UK and he actually… He interacts with a lot of our stuff online, he’s won a bunch of our books when we do book giveaways. 

Daliana Liu: 00:11:06

Yeah. 

Jon Krohn: 00:11:06

So Teslim, thanks for all the engagement. Teslim Adeyanju. And he said, “This is another great guest I would love to listen to again. Meanwhile, I’m curious why Daliana’s show hasn’t been updated since April. We actually miss her good job on the show.” 

Daliana Liu: 00:11:23

Aww. 

Jon Krohn: 00:11:24

So yeah, people are missing you out there, Daliana. But I guess that’s part of the same journey, is it? I think, actually, I heard you talk about this on the Last Week in AI podcast with Andrey Kurenkov, you co-hosted that show a month or so ago, immediately after an episode where I co-hosted. So one of the regular co-hosts, Jeremie, was away. 

Daliana Liu: 00:11:43

Yeah. 

Jon Krohn: 00:11:45

And so, I co-hosted one week, you were co-hosting with Andrey the next week, and one of the things that you mentioned there was that, you realized that you don’t have to be, say, releasing an episode every single week. There’s no law. 

Daliana Liu: 00:12:01

Yeah. 

Jon Krohn: 00:12:01

You don’t have a contractual obligation to do that. 

Daliana Liu: 00:12:05

Yeah. I think around the time I stopped the podcast, there was one reason. I felt burnout. I feel so tired and I want to figure out my focus, the one thing. Another reason is, I joined my career coach, I don’t know, maybe she’s a life coach, cohort. So there was a three-months session and it goes very deep. There’s a lot of exercises, inspires me to add more content to my own course, because there are a lot of things, for example, how to advocate for yourself, talk about promotion. I can literally give people scripts in the cohort, but there is a lot of things. Do you feel comfortable advocating for yourself? What are the limits you have? And through the coaching sessions, I realized I had a lot of fear to do something I really want to do. And then I have a lot of rules that I just assume it’s true in the world. 

00:13:20

For example, if you start a podcast, you have to keep going. Or if you studied science, an engineer in college, or have a PhD, you have to be a data scientist all the time. We have all those rules for ourself. And also, the people during some one-on-one coaching, there are a lot of people who didn’t study data science, but they have been doing practical machine learning, hands-on work. And when I look at their resume, they have done a lot, but people constantly feel… Have this not enough feeling. And I know a lot of time when we have that feeling, some people call it imposter syndrome. And I think it’s very common in data science practitioners, because it’s slightly different from software engineering; it’s not like you build something, it works. A lot of times, it doesn’t work. And even if it work, it would never be 100%.

00:14:29

There are always this 1% you can improve, and your manager might ask you, “Oh, what if we do this?” There’s always a lot of, “What if?” Your stakeholder have a lot of ad hoc requests. So, a lot of data scientists, they’re also scared to ask for a promotion. There are so many factors, so I can just list a few. A lot of us grew up from a culture where you don’t advocate for yourself, you wait for the senior people to promote you. And then that culture, that mindset, come along with us in the professional world. Or something I realize is, I think, for example, myself, I had a pretty happy childhood. During the coaching session, we talk a lot about healing work and I felt I don’t have anything to heal from. I don’t have any trauma. But what I learned is, we all have those wounds from childhood or when we were a teenager, when we were very vulnerable, very sensitive.

00:15:41

Maybe the first time you gave a presentation, your parents or the teacher criticized you. Or the first time you ask a stupid question in a classroom, the girl or the boy you had a crush on, they laughed at you. And those events can be like a wound for you in the future, maybe consciously or subconsciously, you feel scared to speak up during meetings. I remember when I was maybe fourth year, when I was at Amazon, that was before I got promoted, I was asking my manager for feedback. And then the feedback he gave me really surprised me, because I thought he was going to say something about technical skills. And he said, “Well, Daliana, most of the time when you voice your opinions, you are right, but a lot of time, you’re quiet, you are timid.”

00:16:50

So that feedback shook me a little bit, because I thought, “Well, I gave a lot of Amazon employees, engineers, product managers…” I used to lead trainings on machine learning and AB testing decision makings, I’m not afraid to be in front of an audience and I can give presentation, I can be funny, but I would never associate myself to someone called timid. And then what I noticed is, it’s not about being able to speak, it’s about, are you able to disagree with people? Do you feel comfortable to challenge someone’s opinion? Maybe this person is more senior than you. So, I think after that, I try to just have more awareness if I have some opinion, not to suppress my opinion, try to speak up more in meetings. Also, sometime even in personal relationships. If a friend… It’s really hard, especially when it’s a friend. If someone does something that make you feel uncomfortable, are you able to tell them? Or a coworker, for example, this happened to me, reschedule meetings 10 times. 

00:18:14

I feel this person doesn’t respect me, but I also want to be nice. Those are the moment we need to stand up for ourself. So, I forgot what I was talking about. I think going back to why I took a break, this career coaching journey, I noticed I had a lot of those type of fears. I didn’t really believe, “Oh, I can transition from a podcast host to a career coach.” And just in my mind, I have that framework for myself, I’m a technical person, I’m a data scientist, I’m a technical podcast host. And although I have the desire to help people and even through content, I get a lot of people’s DMs. I always screenshot, put them in a folder in my phone, so sometimes when I feel less motivated, I look at those. I know I can do it, but sometimes, now when I announce it to the world, it seems so natural to my audience, to my friends, but it was just myself, put myself in those cage. 

00:19:30

And yeah, I think a lot of times when people are still in a data science or machine learning role, when they think about, “Oh, can I do more GenAI stuff? I didn’t learn engineering, I did not have a degree in AI.” Or maybe I want to be a product manager, or I also have friends become a startup founder, or someone completely left data scientists to do real estate and they’re happy. So, I just feel there are so many stories I can tell and then there are so many things I learned I can share with other people, to empower them to choose a career path that works for me, advocate for themself, to give them that courage and strategies. 

Jon Krohn: 00:20:22

This episode of Super Data Science is brought to you by AWS Trainium and Inferentia, the ideal accelerators for generative AI. AWS Trainium and Inferentia chips are purpose-built by AWS to train and deploy large scale models. Whether you are building with large language models or latent diffusion models, you no longer have to choose between optimizing performance or lowering costs. Learn more about how you can save up to 50% on training costs and up to 40% on inference costs with these high performance accelerators. We have all the links for getting started right away in the show notes. Awesome. Now, back to our show.

00:21:02

Yeah, I think that’s great. So I think to recap two of the main things you’re saying here are that, there are these two common career mindsets that I certainly experience a lot. I’m sure a lot of our listeners are impacted by maybe both of these to a huge amount. And the first one is that we assume lots of rules. 

Daliana Liu: 00:21:21

Yeah. 

Jon Krohn: 00:21:23

And I absolutely do that. We do this podcast twice a week, every single week of the year, and there have been periods where I’m like, “This is crazy. Can I have a summer break? Or a Christmas break or something?” And we’ll see if maybe someday, somehow I’ll figure out how to shake that rule. For now, I’m okay with it. And actually, I’m always really excited right now to be recording the two episodes every week, but there have been points. We were talking before we started recording, when we started finally monetizing this show after five years of running the show. Three years ago, we started to monetize it with sponsor messages, and the first year I was finding the sponsorship myself to test it out. 

00:22:09

And like you and I were discussing before we pressed the record button, some of that stuff was really helpful. I learned a lot about negotiating and pricing, building packages. That was really cool, but I was also super burned out with a full-time job as a co-founder of a tech company, finding sponsorship for the show, and hosting the show is too much. And so, luckily now we have Natalie on the show who does that. Most of her time is spent… She works full-time on the show and most of that time is spent on sponsorship. It’s a huge thing. Anyway. So yeah, we assume these rules around how we need to be living our life. 

Daliana Liu: 00:22:57

Yeah. 

Jon Krohn: 00:22:57

I liked the one that you say there about, if you have a technical degree, you assume that you need to keep doing something technical. 

Daliana Liu: 00:23:04

Yeah. 

Jon Krohn: 00:23:04

And I’ve experienced shifts like that in my life. My undergrad was in science, and so, when I started my PhD, I had this idea in my head, it’s a science PhD as well, and I was like, “Okay, well…” At that time, data science wasn’t a career, when I started my PhD. 

Daliana Liu: 00:23:22

Yeah. 

Jon Krohn: 00:23:22

Even when I finished it, I hadn’t heard of it. 

Daliana Liu: 00:23:23

No. 

Jon Krohn: 00:23:25

But certainly, at the beginning, it didn’t occur to me that there was a career like data science, but that probably would’ve been one of the things that I would’ve thought, “Okay, if I’m doing a PhD, applying machine learning to biological data, data science is probably one of the careers.” So that’s something I would certainly think today if I was starting a PhD. But I was thinking in my head like, “Okay, I could work for a pharmaceutical company, or maybe I could do my MD after my PhD and be a medical researcher.”

Daliana Liu: 00:23:50

Yeah. 

Jon Krohn: 00:23:51

But it seemed to me like there were very few options out there. And a really interesting, mind opening experience for me was, doing my PhD at Oxford, there’s lots of companies, big banks, the big consulting firms like Bain, McKinsey, BCG, those kinds of companies, they’re all like, “Oh, we’d love people like you to come work with us.” And that opened my mind to like, “Oh, I could kind of be doing anything. I could be doing commercial things, like being numerate and having programming skills. This is useful, increasingly, in basically any industry.” 

Daliana Liu: 00:24:25

Yeah. 

Jon Krohn: 00:24:25

And so, you have your pick as a technical person, but even then, it expands beyond that. Like you’re saying, if you figure out through the things that you’re doing, through talking to Uber drivers, Daliana. If you figure out, through talking to them, that, “Wow, something that I’m really excited about is helping people with their careers,” there’s no reason why you have to feel these rules. And anyway, I could go into a million examples myself as well, where you feel constrained, you feel like you must go in a certain path like, “Oh, I’ve come this far, I’ve got to keep going.” Yeah. So that was the first thing that you were talking about there. 

00:25:05

The other second, I think, really big key thing that you were talking about there, is this idea of wounds, where you have these things. And like you, I had a super happy childhood, I don’t feel like I have any trauma to get over. There are interesting things. When you were talking about that, I was reminded of how, in my undergrad, I worked extremely hard. I wasn’t a very hardworking high school student, but when I was in my undergrad, I worked very hard. And there was this one comment, one of my best friends from high school said to me at the beginning of university, he was like, “You’ll never get into med school.” 

Daliana Liu: 00:25:43

Wow.

Jon Krohn: 00:25:43

He was just like, “You’re not that kind of… You don’t work hard enough. You can never get into med school. That’s not something that’s an option for you.” 

Daliana Liu: 00:25:51

Yeah. 

Jon Krohn: 00:25:51

And so, when I was working late or working on weekends, when I didn’t feel like going anymore, it was so interesting, him saying that in my head, I was like, “I’m going to keep studying.” And then eventually, I got myself in a position by the end of my undergrad where I definitely could have gotten into med school.

Daliana Liu: 00:26:11

Yeah. 

Jon Krohn: 00:26:12

It’s interesting how these… That person would never remember that they said that. It was such a random, offhand comment, and it wasn’t intended to be mean at all. It was factual, based on what he knew about me. 

Daliana Liu: 00:26:24

Yeah. And it motivated you. So at that time, that comment served that purpose. But what got you there, right now, if you think about that comment, if you still give that common energy, it’s not going to help you with your future career. That’s going to make you feel… Basically, that’s another assumption. If I don’t work super hard, if I don’t burn myself out, I won’t achieve the results I want. Sometimes we also associate with the amount of work we produce and our self-worth, our job title. Yeah, I think you brought a very good point. I think a lot of time, it’s not black and right. What is it? Not black and right, black and white. 

Jon Krohn: 00:27:16

Black and white. 

Daliana Liu: 00:27:16

[inaudible 00:27:17] 

Jon Krohn: 00:27:18

Something I wanted to say is that, based on your LinkedIn profile, I don’t know this about you for sure, but it seems like you grew up in China, you lived in China until the end of your Bachelor’s degree, and then it looks like you came over to the US to UC Irvine to do a Master’s. 

Daliana Liu: 00:27:31

Yeah. 

Jon Krohn: 00:27:32

Is that right? That kind of timeline?

Daliana Liu: 00:27:33

Yes. 

Jon Krohn: 00:27:34

And so, first of all, your English is unbelievably good. And then the second thing is, that you do amazingly well, is you are so funny, you said it earlier, how in meetings or in trainings that you’re doing at AWS on machine learning for people, how you can be funny. And you’re very funny. And that, to me, is also… That is one of the things that shows that somebody has a really great command of a language when you can be funny. I don’t think I… There’s some other languages that I can speak a little bit, but there’s no way I could be funny in them. 

Daliana Liu: 00:28:05

Oh, thanks.

Jon Krohn: 00:28:06

Maybe I’m not funny in English. 

Daliana Liu: 00:28:11

Yeah, maybe I can share a little bit. I think when I first moved here in my first job, I had a position in a small, medium-sized company in LA. So I was their first business intelligence data analyst. I think that was the title, basically. Maybe now it’s a data scientist role, a generalist. And I noticed I need to give presentations to a lot of people in marketing and sales, and sometimes they don’t understand me. So then I realized, “Oh, I need to speak better English, to work on my accent.” So I think maybe that was the first time I ever invested in a non-technical, personal development. So I follow this English YouTuber, I think she still runs it, called Rachel’s English. And she has a partner, they give people English accent lessons. So I hired this guy, I think he went to Harvard, has some degree in acting. So I would record myself, he gave me critiques. I think that’s probably one of the very high ROI investment that I had. Once I learned about those accent, and then also helped me better understand other people. And another unconventional self-growth thing I did was actually when I lived in Seattle I took a comedy class, because I realized if you can make people laugh in presentation, it’s easier for them to learn. It also builds a better rapport with people. And in my class, I think most people are native speakers.

00:30:02

And then in the final quote unquote presentation, you actually have to go to a local theater in Seattle and we all sell tickets to our friends so people are paying and you’re performing. So I remember I practiced really hard and I didn’t need my notes. And then there are people after the show tell me I’m funny. So that gave me a lot of confidence, and I feel those are the things if you say, “Hey, how to grow your career,” nobody will say, “Hey, take a comedy class, take a communication course.” But I don’t know how to directly map that to later… For example, I joined the team at AWS where it is customer facing, I’m comfortable talking to people, giving presentations, and now hosting my own podcast. But I feel it’s all looking back, connecting the dots, I think those experiences are actually more valuable maybe compared to a technical course I took. 

Jon Krohn: 00:31:06

If you’re a regular listener, you know that last year I did a European podcast tour interviewing incredible guests in Amsterdam, Paris, and Berlin. While all the guests spoke perfect English, Babbel was invaluable for me to learn and practice Dutch, French and German, enabling me to get directions and order my meals in the local language. Super fun, rewarding, and in some cases an essential skill. Now you can do the same with a special limited time deal. Right now get up to 60% off your Babbel subscription, but only for our listeners at babbel.com/superdata. Get up to 60% off at babbel.com/superdata. Spelled B-A-B-B-E-L dot com slash superdata. Rules and restrictions may apply. 

00:31:53

Nice. So I’m going to really quickly ask you… I’m going to ask you some more career tips that you have for our audience following on some of the things you just said. But really quickly, something that I want to draw a line under is, based on the conversations you and I were having before we started recording, it sounds like you might be getting ready to publish more data scientist show episodes. Is that right? 

Daliana Liu: 00:32:14

I mean, I already promised you I’ll have at least one more episode. To be transparent, I think I will continue to publish podcasts, but I might change the name of the podcast or start a new podcast, just call it Daliana’s Show or something like that. Because what I learned about myself is I might not feel super motivated to be a full-time content creator, but I think content creation is a byproduct to whatever I’m doing. I need to have another thing going on. So I remember, I think, I really enjoyed content creation when I was a data science practitioner. So now I think I would still interview data scientists, I would talk to a lot of hiring managers or maybe other people who successfully transitioned in data science roles, have unconventional career path, maybe from… They had a full-time job, now they become a independent consultant. Maybe they become a writer or they have a few things going on. So I will probably continue the podcast but have a diversified topic, focusing more on personal growth for data science and AI practitioners. 

Jon Krohn: 00:33:43

Nice. Well that makes a lot of sense and I could see how that happens. I guess it makes sense to me that you might want to generalize a little bit away from the data scientist show now that you’re not doing that as a full-time day job and being… I imagine, although it sounds like the coaching that you’re doing right now is particularly helpful to people in technical roles like data science, software developer, those kinds of roles, I can imagine that over time that could broaden and broaden and broaden. So it makes sense.

00:34:20

But yeah, while we are on this Super Data Science podcast, for as long as it is called this… There’s no place to change that, but maybe someday we’ll have to think about that as well. Yeah, so in your experience advising AI companies and coaching data scientists, we already talked about two of them. So maybe you’ve already answered this question, like maybe the rules and dealing with these kinds of wounds or feedback from the past, maybe those are your big challenges and obstacles that you encounter. But I don’t know, maybe you have some others. Are there other common career challenges that you see amongst the data scientists and related professionals that you coach, and how do you guide them over those obstacles? 

Daliana Liu: 00:35:05

Yeah, so for example, a lot of times when people want to explore, or maybe I want to do more machine learning, I want to get into GenAI, the question they ask me, “Oh, what courses should I take? What books should I read?” I think a lot of times people are so deep in their own head, the question they ask might not serve their growth. I think the most important thing is to understand what’s the day-to-day of the role you want to get into. Something simple you can do right away is, for example, I always ask people to research 10 job roles to analyze what is the required field, what are the skills they need? It’s not a nice to have, because the nice to have people just put everything there, and it can range from big tech companies to startups. And also sometimes maybe you can just search… Try to stalk 10 people on LinkedIn with the same title you desire to look at their career path, how did they get there? 

00:36:19

And just talking to people, talking to people in your company, your friends, or do some code outreach on LinkedIn to do a lot of research. And I think the best way to transition is always to do that within your company or in your team when you already have trust built with your manager, stakeholder, or someone going to write a review. So one thing I always talk about in the course and coaching is to build trust. That’s the foundation for your manager to want to give you project with bigger scopes, to help you with promotion, give you a raise or later help you transition into other roles. So I think it’s not about the course, the skills you need to learn, it’s more about understanding the other role. So maybe you already have that skill set. Think about what project do you need to do to get in there.

00:37:23

And another thing I think people don’t think about is a lot of times being a data scientist, machine learning engineer, the title can be very glamorous in the outside, but you don’t know how our project is being evaluated. I can think about the performance metrics, it’s also very important. So for example, there are different teams, they’re focusing on how many models you put into production. They are teams, they actually, they also care about publishing research, patents, paper. And on the day-to-day level, if you work in a large company like Meta, maybe you don’t need to build a model from zero to one. There are already very much mature model you can use. So basically you need to… I joked with my friends, so basically they’re config engineers. You just need to figure out what type of parameters experiments you need to do, and it’s very heavy on the engineering skill. Is that something you really enjoy doing?

00:38:36

Or if you’re in a smaller company, maybe you need to think about the stages of their data product. Do they have enough data, is their data good enough? Are they ready to build their first product? If you really enjoy that from zero to one stage. And then that’s also not enough. A lot of times in my work you might notice there’s a stakeholder from another team. This person later left the company, and then sometime those projects get dropped. What is this team’s relationship with their stakeholders? Do you have funding from your VP or another team? And there’s always so many things beyond the technical skill. You need to understand when you are working the role, especially for data scientists, sometimes your manager is a software engineer or product manager. When you think about the role a team, you need to understand what exactly the dynamic are. Talk to a lot of people.

00:39:49

So especially understand the struggles in this role. And I’m talking about this not in a way to intimidate people to get into data science, machine learning, or transition their career, but I think you need to have a holistic view of a role before you think about how to get there, how to transition. Or sometimes I suggest people to do interviews with some roles that’s not their dream job, so you get practice, you get a feedback from the hiring managers, from the recruiter. So now what kind of interview question do you need to practice instead of just in your own imagination. Think about how to grow. So that’s one thing.

00:40:41

If we have time, there’s another thing. I think a lot of times data scientists and machine learning engineer, they are artists. So what is artist? Artists create art from their heart. They want to perfect that craft. They want to improve the performance from 90% to 91. They want to build a perfect platform that maybe in the future other people can use. I think those desire to be a perfectionist is great, but in a lot of times you work for a company, your team think about there is a timeline and there’s conversations about cost. So maybe improve that model by 1 or 2% is not important, but maybe you need to think about comparing costs. Just example, maybe OpenAI, Anthropic, which one is cheaper when you call those APIs? Which one can reduce latency. 

00:41:50

So a lot of those things and go into the engineering aspect of things. And sometimes you also need to think like a product manager. So it doesn’t matter how good your model is. If you’re a stakeholder, especially if they’re non-tech, if they don’t understand your model, they don’t trust you, they don’t trust the model, they’re not going to use it. If they don’t use it, you don’t have impact, you don’t have anything to put on your resume when you look for a job or when you want to have that promotion conversation. So I think data science engineers need to get out their head a little bit to really think about how… Actually not at the end of their project, in the beginning, to think about how this model is going to be used. 

00:42:42

And when you deliver this model to your stakeholder, do you know your stakeholder has stakeholders? If you can make your stakeholders life easy, if you can understand what metric your stakeholder’s stakeholder care about, then it’s easier for you to get buy-in from your stakeholder. And also sometimes maybe machine learning isn’t the solution, or sometimes it’s a dashboard. Maybe you need to spend more time to collecting data. I think it’s also very interesting, as a data scientist, I think sometimes philosophical. You need to be in the craft, but you also need to have this non-attachment to the craft, because sometimes maybe the best decision is not to launch the model. 

00:43:33

And if you’re in a team, do you have a manager that understands this type of situation and provide a psychological safety for you to try things or understand sometimes things don’t work. Or when an A/B testing failed, when this feature is not launched, what can you learn? What other things you can get from this project that will still be useful for your team? So I think from this artist mindset to think like a business owner, think like a product manager, think about time, think about cost. And sometimes, especially with a hallucination of language model, think about your company, your team’s, legal, PR, security. So I think you have to become a… Have this almost entrepreneurial mindset to be a good data scientist. 

Jon Krohn: 00:44:45

Many decisions businesses face are massive, complex, and heavily constrained. In these scenarios, mathematical optimization is often the best tool for the job, and Gurobi, which is trusted by many of the world’s leading enterprises, is the go-to provider for fast, at-scale optimization. While filming Episode #723 last year, I had my mind blown about the wide range of scenarios where optimization is the right solution for the job. Check out Episode #723 as well as the introductory resources for data scientists at Gurobi.com to get up to speed on when optimization is ideal, and add this uniquely powerful tool to your data-science toolkit. Then, coming up in August, we’ll have a second episode on optimization with even more tips and tricks from Gurobi guru Jerry Yurchisin — hope to catch you then.

00:45:34

That makes so much sense to me. So to recap some of these key career mindsets that you see, the challenges that they encounter, and the solutions that you have, there’s the… I already recapped on the assumption of rules that people have. We talked about the wounds that people have from feedback. But since then some of the great tips that you’ve given are around what the day-to-day of your dream role is, so understanding that day-to-day role. So titles, something like data scientist, that could mean so many different things. And I have a follow-up question for you on that in a second. So really understanding what is going to happen in this role is key. And a lot of that can have to do with the stage of the company that you’re at. You gave some great examples there when you were explaining that. 

00:46:21

Another big thing is thinking about what the commercial impact of a project will be. So being entrepreneurial, not just being technical, but thinking about costs, legal issues, security issues, and then having that whole entrepreneurial mindset lead to great data science developments that can be, say, incorporated into a product or into a business and drive commercial results. And that is the kind of stuff, more than anything, that is going to be key to future promotions or landing your next big job. Yeah, I agree. Cool. All right, so I just said that I was going to have a follow-up question, so my follow-up question is that when… A data scientist, any stage in their career, they could be early, they could be late, the field moves so quickly. Maybe they aren’t even a data scientist yet. They’re listening to the show and they’re thinking about becoming a data scientist in the future. 

00:47:17

The field moves so quickly, the title data scientist tells you very little about what you’re going to be doing in that role. It could involve any number of building a model from scratch. That’s like the dream that a lot of people think they’re going to be doing in data science. But probably doesn’t often happen. A lot of it is data cleaning, data engineering, machine learning engineering, getting things into production, ML ops. There’s so many different things. Creating reports, creating dashboards. There’s hundreds of different possible kinds of jobs in this title data scientist. 

00:47:53

So given that, there’s a couple of… I expect these kinds of things that you encounter in your career trainings. What advice do you give to people when they’re unsure what direction to take? So they’re like, “Okay, I want to be a data scientist.” But then how do you… I want to be a data scientist or I already am a data scientist, but there’s these other things that I could be doing. When somebody’s unsure of what direction to take as, say, a data scientist or a machine learning engineer or a software engineer, how do you help them find a niche that they’re passionate about and that is aligned with their value and where they’d like to go in their career? 

Daliana Liu: 00:48:44

Yeah, I think it’s interesting you mentioned niche. I think a lot of times when people tell me they want to be a data science or machine learning engineer, it might not be what it really wants. Sometime they’re following this hype. It’s not really their strength or they’re only aware of a few job options. So for example, later I talked to my friends in recruiting, there’s also data science in finance. Their job titles are like quant. Or even a trader, if you are very quick at math and statistics, you can be a trader. And those roles has probably a little bit more stress, but you probably make double or triple a data scientist salary. So a lot of people are not aware of those roles. So usually I will… Going back to this person’s background. So another example is someone had a one-on-one with me. He had experience in doing government contracts. And I know to do those roles you need a security clearance, and a lot of people who are immigrants cannot do those type of roles. 

00:50:09

So for this person to transition, there is some niche, maybe you can look for data scientists in public sectors. So all those large companies like Google, Amazon, IBM probably have those sectors. So usually it’s personalized. I want to find out would it have done before, how can they leverage their past in your future roles if they are not resisting that part? Another thing is, I’m going one step back, I always ask them, “Why do you want to transition?” A lot of people say, “Oh, I’m passionate about model and stuff.” Sometimes yeah, it’s like the story you tell recruiters, but when I have conversation with people I coach, it can be honest with me. Sometimes, “I just want to make more money,” or, “I want that title. I think it’s cool.” “My immigrant parents want me to have that job.” 

00:51:04

And some people, they don’t have a strong desire to have to be in that role, they’re more flexible. And I think they’re different stage in people’s life. Maybe in this stage of your life you do need to make more money, or maybe in the current stage of your life your parents’ approval or your spouse approval is important for you. And I think that’s okay. I think it’s to be honest with yourself. If you want to pursue the job that make those money and I can help you with that. And then I will try to get them to be honest about what they really want and their strengths and weakness. Again, a lot of people go into the route of what course do I need to study? I’ll ask them to think about understanding the day-to-day. And if you do need to interview an external company, understand what is going to be asked in the interview. 

00:52:05

So one shift I noticed is for data scientists and machine learning engineer, there are more lead code questions in the interview. Because if you think about it… I joke about it with my friend, AI engineer is basically API engineer, API ensembler. Of course there are more things under the hood. You need to understand those metrics, how to evaluate them. But a lot of times the models become better and better. What you really do is just put them together. Also similar for data scientist. No one is creating a regression model or tree or neural network from scratch anymore. You can do that when you’re learning. I think it’s helpful for you to learn the skills. But in the reality we don’t do that anymore. So are you familiar with the APIs, the tools, if you think about PyTorch, TensorFlow. 

00:53:10

Or there are a lot of MLOps tools you can also just config engineering, run experiments. So I think focusing on engineering skills is important, but that comes after you realize what is role actually is required. And I think also understand after three or five years, where do you want to go? Do you want to be a specialist? Do you want to find your niche? Or do you want to be a generalist? I think unless you have a strong desire in a niche, you can go into it. But I think right now, especially if you work in the industry, our work is focused on the problem. I know a lot of my coworkers previously in their PhD, they’re doing research on NLP or computer vision, but now they’re doing something completely different. 

00:54:20

I think it’s important to have a sense of curiosity, want to learn. I think it’s okay to hold on to the area you’re very interested in, but also open to other roles. I think you don’t have to be a generalist in a way that you’re doing all type of engineering or web design. But I think it’s helpful to… Sometimes maybe it’s easier for your stakeholders to play with a model if you can create a little demo using, I don’t know, Streamlit or- 

Jon Krohn: 00:54:57

Yeah, Gradio. 

Daliana Liu: 00:54:58

Yeah, things like that. So I would say if you’re confused in your career, I think based on your past and where I want to go, have a few options, but those options better not be too far apart. I think easier for you to get into I think is the best way to go. Also, I noticed sometimes people are like, “Okay, I have done my research, I talked to people, I want to be a product manager. But maybe the best way for me to do is to be a data scientist, product data scientist, and then transition.” So a lot of people, they kind of want to zigzag into another role, but I think that can waste you a lot of time. If you’re already certain you want to go into the role I think the best way is to… It’s some first principle thinking to think about how do you get into that role directly?

00:56:04

Because when you get into an intermediate role, you’re not really present, you don’t really enjoy that, you think about how to get there. But this role’s day-to-day can take a toll on your energy and then you might not have time to prepare for another role. So when I got into Amazon, my title was business intelligence/statistician. I think that’s basically what data scientists do, but there wasn’t the title of data scientist. And in my first job before Amazon, I wasn’t specialized in A/B testing, but I was open to the opportunity. And then over the year when I got into it, I thought it was really fun to advise product managers, engineers to make decisions, so I stumbled into this niche. 

00:56:56

And later on I got curious about machine learning, I want to learn more about machine learning and deep learning, so I made that transition. And then now when I look at my career, okay, I’ve done a lot of experimentation, product data science. I also have done machine learning, and that make me a generalist. So a lot of things, it’s not… Things happen, your interest change, it’s not exactly as you planned. So I think I also don’t plan too much ahead, be flexible. I think that’s also very important. 

Jon Krohn: 00:57:35

Nice, yeah. So to recap on some of the key points that you made there. So my question was about how you find your niche under that big data science title umbrella, and you started off by answering that the question that people need to be asking is what they really want. So being honest about that, because like you say, a lot of people, they have their job interview answer prepared about what they’re passionate about, when in reality they just want more money or they want their parental approval or their spousal approval or whatever. So it sounds like that kind of honesty and realizing that there could be related kinds of jobs, like you mentioned quants there. 

00:58:16

But maybe it even means that somebody should be thinking about career coaching even though they have a technical background. There’s a huge range of possibilities out there, so you don’t do yourself a service by having a rule that says, “Oh, data scientist is, whatever, the sexiest job of the century. That’s what I want.” Maybe when you’re actually doing it isn’t what it seems. Now one thing that’s interesting is that you said that nobody, you said nobody actually makes models anymore, like regression models or random force. It is interesting that that is still… At my company, Nebula, that is something that we do still do. We do make models from scratch for certain use cases. 

Daliana Liu: 00:59:01

Oh. No, sorry, let me clarify. We still use regression. I suppose what I’m saying is, we are not coding the algorithm from- 

Jon Krohn: 00:59:17

Oh, right, you use the Scikit-learn API or Python. 

Daliana Liu: 00:59:17

I’m glad you clarified that, yeah. So, actually, that’s a big confusion right now in the market. People are like, “Oh, everything is GenAI,” because those content get hyped up in social media. And as a content creator, now I know a lot of people, content creator create those content because it’ll get them views. But in the industry still, a lot of people are using SG Boost, a logistic regression. And I talked to a few friends, they work in more traditional industry. They’re not using any LLMs, especially your customer is a business or 2B business. I think if you’re 2C business, it makes sense. You can create some chat bots for your customer if you have a lot of documents. But to see if most of your feature are numerical or those use cases where requires a lot of transparency, inter model interoperability, people are still using, I guess now people say traditional ML or deep learning. 

Jon Krohn: 01:00:28

Yeah, yeah, yeah, exactly. Okay, cool. I’m glad we clarified that because, yeah, then now we’re definitely on the same page. Yeah, it is interesting how, because of all the hype around GenAI, there probably is an assumption that that is a lot of what data scientists do. Now, if you are a data scientist working with GenAI, that is a scenario where you probably are mostly actually using already trained foundation models, certainly to prototype. There could be some circumstances where it makes sense to download Llama 3 model weights and fine tune that using LoRA or some other parameter efficient technique and actually run an LLM on your own infrastructure. But most of the time you don’t need that. Most of the time you can be relying on an existing foundation model and the tooling just gets easier and easier and easier every month to the middle layer to if you are, as you say, so Daliana used the term 2B or 2C, and that’s like business to business or business to consumer.

01:01:29

So a B2B business is serving other businesses as the primary customer. And a B2C business, that’s the direct to consumer product. And so it is interesting in, you spent a lot of time in the Bay Area and so in the Bay Area you only build B2B SaaS companies, that’s like it. But there’s this small sliver of very brave entrepreneurs that are trying to build, because if you can get a B2C business to work, those can end up being some of the most well defended in terms of having a moat companies out there that end up being the most valuable. So if you think about Facebook, Instagram, WhatsApp, LinkedIn, these are platforms that they were created initially to serve a consumer as opposed to a big enterprise client. And then once you have all of these users and you have all these data and you have this huge moat, then you start figuring out how to monetize.

01:02:38

Anyway, I’m off on a tangent, but yeah, so most companies are B2B. And yeah, so you’re saying with the GenAI thing, yeah, if you’re building B2B, most of that work still is probably going to be not a generative AI. There’s probably somebody or many people in management of the company that is saying, “Why isn’t the data science team working on GenAI?” And maybe you do some small pet projects, some prototyping. But because of the cost, because of legal issues, hallucinations, it can end up being that you don’t maybe move as quickly with B2B applications of GenAI. Anyway, off on some tangents here.

01:03:20

All right, so one last question for you in this direct topic area around the data science title and how it can be so nebulous. So data science is constantly evolving and that’s part of also what makes the data science title tough because if you are in a data science role for many years, what you’re doing three years from when you started the job is likely to be very different because the underlying tools change. Maybe now you are fully immersed in building GenAI things which you wouldn’t have dreamed was maybe even possible in your lifetime when you started the data science role. And now you’re like, “Wow, I can just have my users put a natural language question in and have any answer come back, and 99% of the time it’s exactly what they wanted and it’s calling my APIs.” And so these kinds of crazy completely paradigm changing things can happen in a data science career. 

01:04:14

And because of this fast evolution, it can be fertile territory for things like imposter syndrome, which you mentioned sometime ago, for distraction, for just how do you know if you are following the right thing? You could end up spending so much time learning some programming language or some tool and because it seems like that’s going to be the up and coming thing, but then something else comes along. This is an unfair example to TensorFlow because TensorFlow is still quite widely used, but what if you’re like, “Wow, deep learning is going to be big. You spend all this time learning. Actually, a really good example is learning TensorFlow 1. Before TensorFlow 2, TensorFlow was like this very cumbersome, very complex thing. You’re working at the graph level and so you can invest a whole bunch of time becoming expert at that, and then they release TensorFlow 2 and all of that is an abstracted way. And so maybe it ends up being helpful to you in some indirect way, but you feel like you’ve wasted all this time. So imposter syndrome, distraction, and of course burnout, which we’ve already talked about earlier because of this constant evolution, this feeling like no matter how fast you move, you are seeing things move faster. 

01:05:36

And this can happen with data science skills. I think it can also happen if you’re more entrepreneurially minded, which could be internally, that could be the data scientist entrepreneur mindset internally that you were talking about earlier. Or you could be literally an entrepreneur. For me, as a co-founder of a tech company, I’m like, “Wow, we need to be taking advantage of these GenAI things.” And I see every day these crazy valuations or crazy amounts of ARR that somebody else managed to capitalize on say this GenAI thing faster than me or more effectively than me. And I have this constant feeling of being so far behind. And so imposter syndrome, distraction, burnout, how can the people that you mentor and how can our listeners, I guess in the same question, how can we build confidence and resilience through all of this change and all this challenging learning? 

Daliana Liu: 01:06:40

Yeah, that’s a really good question. And to comment on the learning tools and then feel you wasted your time. I remember when I just started joined the machine learning solutions lab in AWS I took some courses in PyTorch, TensorFlow and AWS will use Glue on CV. And then I also feel, oh, did I wasted my time? But later you read your coworkers code, do you understand what is it about? And then I think because we are in science engineer, we have this desire to optimize things, right? Especially what I noticed is I think engineers really like to make their own coffee, make their own coffee art. I think it’s good to have the desire to optimize things to be productive, but if you think about your career in the life, it’s rarely a linear growth. 

01:07:42

I think first is to let go of the desire, oh, I have the shortest path, the quickest path to get a promotion. I think that’s also the quickest path to get burnout. And if sometimes I think there’s no optimized way to learn. I think it’s learn whatever is needed for your current project. And I know now there’s so many papers coming out, there are new tools every single day. And you don’t need to be on forefront of using all the tools, understand the papers. You know who needs to do that? Only those tech YouTubers need to do that. They need to talk about the newest thing. But if you work for a company, that’s me and then some of my coworkers adopt it’s okay to be a little bit late. So whatever is working right now, you can continue to use it. But always be open to keep an eye on the new tools. If you think about putting things, especially in production, you want to use a tool produced by a company that has been there for a while. 

01:08:58

If you think about SQL, probably not going to go anywhere. That’s pretty robust. And when you think about what things to learn, again, coming back to your goal, your short-term goal, your long-term goal, is this tool needed in your current project? And also I think sometimes it is useful to be curious to want to adopt a new tool. So for example, use the best way to get into GenAI, learn build LLM application, I think is to build a small app for yourself using some OpenAI API. Just think about how to apply those things instead of focusing on those noise. And yeah, it’s never enough I think, but if you have a goal to think about what you want to achieve in the next three months, and I also think it’s in the Bay Area, people talk about anti-goal. I think how I think about it’s just have a list of things you can’t learn. 

01:10:05

I remember when I was learning a computer vision for a project at that time, there are also a lot of NLP things coming out. I think, yeah, I am curious. Maybe how I learned those I just watch a short YouTube video on it or I listen to John explain it on Super Data Science podcast. I don’t have to read a textbook, so I preserve my energy to what I need to do at that moment. So going back to your goal and then just know that no one, no researcher is always reading new things, trying new things. Most of the things are going to be outdated in six months. It’s not very useful. And for that feeling of not enough, I think a lot of times what we need to develop also is the self-trust, the self-confidence. Previously, we’ve talked a little bit about those wounds, some other people’s comment. 

01:11:12

And then I have this idea I’m always very excited to share. I never talk about it. If you think about LSTM or some language models, they always have those forget gate. They will retain only the most relevant information, put it on this highway in each cell and then retain that because that’s productive. But for human, we retain information, not because it’s useful for us. Sometimes it’s because it triggers a very intense emotion. Sometimes it’s fear, shame, sadness, and then we carry that information, right? Also, when you first meet someone, the first impression, if you think about it’s crazy. It’s assigned a very irrational weight in our head. I really wish in my head there could be something I can rebalance that weight of some first impression of someone. I can update this impression because people change. Or I can have my own forget gate. I forget those quote-unquote wounds I have. I think in this way the language models are more intelligent than us. 

01:12:30

So I think for us it is to, I think for me, journaling is very helpful when those moment of doubt we have. Maybe we cannot get rid of the information like having a forget gate. But having the awareness, oh, this fear, I’m in a better place right now. I have a degree. I have done those projects. I think having a documentation, I can call it a brag log. Every time your manager, your customer give you some positive feedback or you finish the successful project, or it could be just this very small bug that took you two days, you finally debugged it, put it in a document. So later those are the information you want to carry on in your LSTM cell. You don’t want to forget. But I think a lot of times we don’t give us credit or we are very self-critical, which if you think about it, it’s not fair. You’re not giving those positive feedback a proportional weight that’s helpful for your future success.

01:13:50

And another story, if I have time I’d like to share, is when I was in AWS, there was a project that has a very unique goal, that was to prove to our customer the art of possible of machine learning. So the goal was to build a demo because we want to build more machine learning project for them, but they didn’t really have buy-in of investing in our services in compute. So after I built, me and my coworker built a solution for them, later on, I think I want to write my maybe annual review. I wasn’t sure whether that project I implemented whether engineering a production, but I wasn’t sure whether it was used, because I want to get the business metrics to support this project. 

01:14:53

And then I feel I have the imposter syndrome, the doubts kicking, “Oh, if the customer is not using it, did I actually make an impact? Is it because I didn’t do enough so they’re not actually using it?” And then my manager told me something I often try to remember, and she said, “The goal of that project was to show them the possibility we were to build a model. I think it was already great. We tried to put it into the production so you reach this goal and it was good enough.” And I think hearing that from a manager was very reassuring. And then it made me feel not just feel good, it made me realize, okay, I did achieve the success for that project.

01:15:51

Different project have different goals, and I think a lot of times when we evaluate ourself, we don’t give ourself that credit. Maybe your manager is very critical. Maybe you are very critical for yourself. I think to have confidence, to have the courage to later advocate for yourself to take on bigger project is to remember have this brag log of what you have achieved and also remember the goal of the project. It doesn’t have to be perfect, but maybe the goal of the project is to improve a model from 60% to 70%, right? 70% is not a super high percentage of accuracy, but it’s good enough. You beat the status quo. And then that’s a win that you need to celebrate.

01:16:47

I remember when I just started to building a machine learning model, I was just looking Googling Reddit, Stack Overflow, “What percentage is good enough?” And then I realized there’s no universal answer. You have to know having a baseline, it’s so important. Know the baseline so you know whether you have succeeded or not. I think a lot of us listening to the podcast probably always want to improve ourself. I think it’s good that you always want more. But I think at the same time you can give your credit and then celebrate the success and then feel worthy of wanting to have more, be rewarded more. 

Jon Krohn: 01:17:39

Yeah, lots of great useful nuggets in there for people of any career though. Also some ones that are really great for people in data science specifically, like that idea of what’s a good accuracy or what’s a good area under the curve, the ROC curve for this particular binary classification model or whatever. And there’s no right answer because some problems are way harder than others. Some of them maybe it’s going to be trivially easy to have your binary classifier be perfect essentially every time. And then others situations where that same, where a binary classifier, it’s like if you want to build an algorithm that’s going to predict whether this stock is going to go up tomorrow or go down tomorrow, it’s like if that can work 52% of the time you might be a billionaire. 

01:18:26

If you can scale that up in a hedge fund, those are the kinds of things you’re looking for. You’re looking for just tiny, tiny, tiny little bits of edge because so many other people in the market are already taking advantage of all the signals that are out there. So yeah, really interesting that you said those. Something that I wanted to get back to that we were talking about a lot in the first part of the episode. In the first part of the episode, we were talking a lot about your own content creation journey, and we also talked about your Maven course. And my apologies, I only have written down the title of what it used to be like Advance Your Data Science Career With Proven Strategy. Remind me what the new Maven title is?

Daliana Liu: 01:19:07

Survive And Thrive In Data Science. 

Jon Krohn: 01:19:13

Survive And Thrive In Data Science. Yeah, yeah, yeah. So as part of that curriculum, a lot of what you teach is related to this idea of being entrepreneurial, so helping data scientists and probably related kinds of professionals, ML engineers, software developers to build an influence, to show impact, to sell projects internally. And so there’s a lot of content out there that focus is on hard skills and it seems like these kinds of trainings, this content on political maneuvering and marketing yourself effectively internally, it’s huge for career success as I’m sure you know as a career coach. And I agree with completely.

01:19:58

And so I’d love to dig into that a little bit more here. I guess, what are some of the kinds of key tips that you have for people if they want to be with some of these things we already talked about in your earlier answers? So things like being mindful about cost, being mindful about legal issues, security issues, all those kinds of things. Things like creating a demo in Streamlit or in Gradio. Those kinds of tools make it easier to sell something internally because people can click around and try it and use maybe a simple model that you built to prove some point. When you’re trying to build an MVP model, you can have those MVP models surfaced inside of these apps, inside of these simple apps. Gradio apps can take seconds to build and you could show those off. So those are some of the examples that you’ve already talked about that are great. But I don’t know, I hadn’t explicitly asked you this question of whether you have specific kinds of tips or common mistakes that you see on these kinds of soft skills that are so important, building influence, marketing projects, selling them internally? 

Daliana Liu: 01:21:12

Yeah, you’re talking about building influence internally. Yeah, I think the common mistake is just not trying at all. They usually, a lot of people feel I don’t have anything to showcase or people would think build relationship people is like politics. To me, I think politics is a neutral word. Maybe we don’t like it. We can call it build relationship with people. I think previous I mentioned a little bit what is important. For example, when I was in a team at Amazon, in Seattle, I had a mentor and she just genuinely liked to connect with people. And sometime we think building influence, meaning you have to be loud, you have to be super extroverted, always talk about your achievement, which is off-putting. She’s very low-key. She doesn’t even give a lot of presentations. She still does, but she would network with a principal, engineer or scientist to know what they’re working on. If she’s interested, she will tell them, “Hey, this is my skillset. How about we work on this project together?” 

01:22:41

So sometimes working with a senior or principal, a scientist can help you build more skills or helpful for your promotion if that’s what you care about. So basically she doesn’t wait for her manager to assign a project for her. She would mold her project on her own. Of course, you don’t want to bypass your manager. Of course, I think she would get an okay from her manager when she does this. And also she has the superpower. She became friends with the EA, the assistant of a director. So sometimes if you want to, of course as I see you don’t really get face time from director, but I think some of her project has that visibility. And then the EA would help her get into a meeting. Or sometimes, of course the EA won’t share anything confidential, but if that’s just something okay to share maybe what the director is working on, what his focus and that would be useful for her projects. 

01:24:02

And also sometimes work on a project, maybe your stakeholder is moving to another team and you need to know who is someone going to take over, right? This person also need to be familiar with the metrics. You need to also get support from them. And if you don’t know this, when your current stakeholders left, maybe the new person will abandon the project that you have been working on for the past year. So I think building those relationship, this network at work is very important. And you have to be genuine. I have definitely been on a side when I feel someone asked me to do something for them or act very friendly, but later I found this person just is very, not even transactional, just one-sided want to use me. I help them interview some people, give them some advice on their project, but later on, this person just disappeared or pretend they don’t know me very well. So there is a way to build relationship at work and also be genuine.

01:25:22

And also you need to know that everyone is busy. You don’t always have to network with a director. Sometimes maybe the director is your stakeholder, but the person, maybe there’s a senior engineer on the team that you collaborate with more often. And then just build a relationship with this engineer. When I say build a relationship, maybe just schedule some type of sync up with this person and then also be curious about this person. “Hey, what are the project you are working on?” And know whether they have enough capacity to invest your current project. Or maybe go look at this person’s LinkedIn. What’s their background? Maybe they studied, I don’t know, mechanical engineer. You also studied mechanical engineer and then that’s something you can bond with. 

01:26:16

I think it just, a lot of times people have this fear about networking or they would feel, “Oh, I’m a scientist, I’m engineer. Those things are below me.” They have this ego, this pride. They don’t do this type of things. So beside the things you can do interpersonally, there’s also things you can do, for example, you can say, “Oh, I recently done this project.” It could be very simple. You use your internal tool, you automated some process. Maybe it’s not data science or ML-related at all, but it will help your team’s productivity. Maybe you can ask a manager, “Hey, how about we do a lunch and learn session?” And you can showcase your work or you can submit to internal conference. There are other ways for you to build influence. Or something very simple. Maybe you get a lot of requests from your stakeholder side. You can host a training for them, you can do an office hour. So there are a lot of ways that you can build influence without being, you don’t have to be this social butterfly without being annoying.

Jon Krohn: 01:27:38

Nice. Yeah, those are all great tips. I think the main takeaway that all of us have is become BFFs with executive administrators. That’s the key. Then you’re set. No, you have lots of other great tips in there as well. So being mindful of what your stakeholders’ stakeholders looking for, so that then even if your immediate stakeholder moves somewhere else in the company or leaves the company, your projects are likely to still get their budget and the attention that they deserve. I also like the point there about shaking the pride that some technical people have, that this kind of relationship building is below me. I’m just going to focus on being really good at my technical stuff, and that’s good enough. And yeah, you could be missing out on a lot of opportunity, a lot of influence if you have that kind of pride. So great tips there. 

01:28:30

You’re a content creator, obviously. And you’ve had this really successful podcast, The Data Scientist Show. You have over 250,000 followers on LinkedIn. People obviously really like your content. Do you think it’s important, or what do you think the value is for people to create content when they’re not looking to be a full-time content creator, like you eventually did make that transition, but just, is it useful for people for furthering their career in general, either for getting promotions, or for maybe landing their next job to create content?

Daliana Liu: 01:29:05

Yeah. A lot of people ask me this question. So to simply answer just for promotion, no. Your LinkedIn how many followers. It has nothing to do with my promotion at work. If anything, sometimes I have to be actually more cautious at work to make sure I don’t talk about anything confidential. I need to get the approval of HR. So actually if you’re not careful, what you do externally can create issues with your [inaudible 01:29:40]. 

Jon Krohn: 01:29:40

You could also imagine, I actually thought where you were going to go with that in terms of the being careful, I thought that you were going to say that you have to be careful that your full-time employer doesn’t think that this content creation is a distraction from your job. 

Daliana Liu: 01:29:56

Yeah, definitely. I think it depends on how you position yourself. And if you aspire to be a content creator, you want to have a lot of followers. That’s a different story. Maybe you don’t need to care about what people think. And I think that’s totally fine, but if your goal is to build your personal brand in the industry, I don’t think your goal is to have a lot of followers. I think there are some people who are very respected in the industry. Maybe they just have 1,000, 2,000 followers, but what they really enjoy doing is, they will write blog posts, they will go to a conference. I think there are different things for you to build your personal brand. Maybe you can have a YouTube channel, again, with a goal not to become a YouTuber. You don’t have to update weekly, but you can talk about a side project you have worked on, right?

01:30:56

Maybe you have published it on Medium. So I always encourage people to, if you have a side project, have it on GitHub. If nobody can see it, that means it doesn’t exist. And if you put it on your resume, make sure you have the link, and it’s updated so recruiters can find it. Especially now when people are recruiting people with AI skills, because it’s so new. And some recruiters don’t know what to look for, but if you have built something, they can see, they can play with, it could be a blog post. The simple way for you to output your project, I think it’s very helpful. So I think it’s less about the craft of content creation, the eyeballs you get, whether when people search your name, can they find something that you think is useful for your career?

01:31:54

I think when I started content creation, the goal was a little different. I think it was more like I just have this desire I want to share. I’ve seen so many misinformation about the data science role. So my content I have, I think technical content versus career content in the beginning was like 50/50. But if your goal was to grow your career, I think you can focus on the artifact, creating the artifact that other people can see instead of, you don’t have to constantly be pumping out content. Also people will ask me, “What platform is the best in 2024? Should you do YouTube, or LinkedIn, or Twitter?” I think whatever platform that’s the easiest for you to get started and you can resonate. That’s the best platform. Sometimes you might change midway. Again. I think sometimes the desire to optimize, finding the shortest way, sometimes is counterproductive, using the data scientist mindset of experimentation.

01:33:03

So sometimes I know probably a lot of you listening to this, you have 10 drafts sitting there, and you never had the courage to put it out, because you fear what if nobody reads it? What if nobody likes it? If nobody likes it, who cares, right? And if you think that’s a little awkward on your profile, you can always delete it. You can edit it. I think a lot of the time it’s just, even for me right now, sometimes I still have a lot of fear when I put out content. I think that’s normal. Instead of thinking, “I want this to have a million views,” just think of it’s a little experiment you put out in the world. It’s a little product you put out to gather data. Whether people like or comment, it’s data you can collect. 

01:34:00

And there is more I can talk about. For example, what is the ratio you create for recruiters, or for other people to notice? Or sometimes I think the best content is “selfish” content. I think maybe if you feel at work you have to think about the business impact, but you actually want to be an artist, you just want to create something for yourself, I think content creation is a great outlet. Maybe you just want to write a blog post, talk about the career story or a specific technology that you don’t get to use at work. I think that’s also a great way. Think of it as a way for you to express yourself. Maybe just one person on the other side of the screen saw it and found it useful sends you a DM, I think that’s a great win. You don’t have to be famous. 

01:34:59

So actually recently, I just shut down a website I created. It’s called ensemble.careers. So maybe that’s my first try to be an entrepreneur. I want to create this forum where people can ask career questions. So recently I closed it, because not a lot of people use it. I think I probably have 30 people registered. The post most popular one has just over a thousand views. However, there was one post I talked about, someone asked, “What kind of data scientist should you be?” And I answered in that post. And then Amazon Science, the editor, found that post and then reached out to me, and then wrote a feature story for me. And then now it’s on Amazon Science. So the whole website I created, or the post didn’t have a lot of traffic, but just that one thing was found by the right person. And then that has become a very important part for my public profile. So I think in the beginning, have a sense of play to have fun, not have too much attachment. But I think to have anything put out there is better than having a draft sitting there. 

Jon Krohn: 01:36:31

Yeah, exactly. It’s about starting before you’re ready, and being okay with feedback. And generally, people are nice. I think you might get some people who are mean, but that’s usually more on them than on you, if that’s happening out there. So don’t feel bad about critical things that come out. And mostly, I think publishing on a schedule can also be a good idea. Like you say, once a month on this day, that’s my deadline. And even maybe your first blog post or first LinkedIn post as you start to do this can be, “I am committing to do this, so that on this day of once a month I will create something,” and then you stick to that like it’s your job. And pick something that’s easy. Don’t start by saying, I’m going to make two podcast episodes a week, because it’s not sustainable.

01:37:30

Yeah, so great tips there. Yeah, so I think hopefully if we have listeners out there that are sitting on those 10 drafts, publish them. You’ve got 10 months of content there following this kind of idea. And even if just one person, it can be any small number of people, if they’re positively impacted by that content, it could make a big difference. So it’s not just about how much traffic you get or how many views it gets. And like you say, who knows? One piece of content like you had on that old site gets picked up, and it can make a big difference for you. And also, just putting things in words really does help you figure out where you’re making assumptions, and where you need to maybe shore up your own understanding of something. So it can be useful in its own right, but there’s something about posting it publicly. Like you writing for yourself, it doesn’t have the same, you don’t put the level of rigor into fact checking, or into the quality of your writing than if you’re publishing it publicly. So yeah, that’s all really good tips. 

01:38:36

Daliana, my last topic area for you quickly, and we don’t need to spend a huge amount of time on this, but you obviously see a lot in the coaching that you do. You have a tremendous amount of experience as a data scientist. And so I’m curious where you see the field going. So we’ve talked a lot in this episode about soft skills. Yes, very important. But I’m just curious, in addition to the soft skills, when you’re doing career coaching, are there technical skills that you highly recommend people have in data science? Maybe this is related to what’s coming with generative AI being here, for example, with Agentive AI coming, and becoming more and more important. Are there particular skills, technical skills that you recommend to our listeners? 

Daliana Liu: 01:39:22

Yeah. I think again, previously we mentioned data science roles can be very different in different companies. So it depends on what you do. For example, my friends are building forecasting models, so their work is not highly impacted by generative AI or large language models, although they do use some type of forecasting. You can use transformers. And there are other types of machine learning engineers. Maybe they work for a small, medium-sized company. And maybe they want to take advantage of some LMs, identify some low hanging fruit. Maybe you can build some chatbot, or question answering tools for if you are a 2C business. 

01:40:22

So I think again, the most important thing is to identify the business use case, but you have to understand what are the technology? What’s the possibility out there to identify the use case. So if you are in a more generalist role, and then you feel your company has a lot of documents, you can improve either the business use case. Or I think a lot of times for data scientists it’s to improve the internal process. For example, maybe your stakeholder cut a ticket. Maybe you can create some GPT bot to tag, put it into a certain category. I think right now, a lot of AI/LM tools are more like an intern you have. It can help you solve a lot of problems, create some project, but it’s not good enough to rely on it as an advisor. So you’re still the person to make decisions.

01:41:27

And also, my friends from hiring agencies tell me, so actually people are not hiring the title of AI engineer, because it’s very specific. At end of the day, AI skills is just going to be part of engineering or part of machine learning engineers, but they still want people to have those skills. So when they’re hiring software engineers or machine learning engineers, if you have experience leveraging, say OpenAI API to build an app, knows when to fine tune or use a RAG, that’s also important. Because a lot of times now when companies are hiring people, again, this role is new, this skillset is new. They don’t exactly know what skillset they’re hiring into. So you are an advisor. You need to tell your hiring manager what you think they should do with their data, with their business use case. 

01:42:29

So be familiar with OpenAI or Anthropic, those APIs, fine-tuning. But based on my conversations with practitioners, the RAG use cases of chatbot are more common than fine-tuning, because not every company has their own data, or their data quality may not be good enough. I think sometimes RAG is good enough. And also, when you think about building those AI solutions, it’s connected to a database. So get familiar with some vector database for example, on Pinecone, and those things can be useful. And if you are interviewing for large tech companies, I think for data scientists, machine learning roles requirement for software engineer is higher and higher. So previously, I think people asked questions about data manipulation in Pandas, or maybe like a simple lead code. 

01:43:34

Now I heard at data points, sometimes people will ask Medium, or even hard lead code questions. Which I think makes sense, again because models are becoming better and better. A lot of times, companies are thinking about scaling the solution, reducing the latency. So you need to really be proficient in your software engineering skills. And for product data science, I don’t think it has changed that much. Product data scientists or data analysts, BI type of roles have less requirements for software engineering skills. Some might still ask you lead code, simple questions, but they want to make sure you know SQL, and you can do similar type of data manipulation in pandas. I do have some data points. People tell me the questions about experimentation, AB testing is getting a little bit more difficult. Sometimes it could be causal inference. I think it depends on the company your interviewing. 

01:44:49

Again, if right now your work is pretty demanding, it doesn’t really get into the AI or LM side. Don’t feel anxious to focus on the tools that will make your current work easy, but in your spare time there are so many things to learn. To learn things, it doesn’t mean you have to sign up for a big course. You can read blog posts. Just be aware of the possibility. So whenever you feel this specific tool or technology can be useful in your current workflow. And then you can learn at that time, it’s not going to be late. 

Jon Krohn: 01:45:30

Nice. Yeah. Great answer. You had a lot more detail there than I was necessarily anticipating, but I should have anticipated that, because you are a career coach. So like you say, you get these data points, and they’re really useful. Some of the key ones that I pulled out from there are, if you want to, having experience with generative AI APIs can be something that’s useful. Building a chatbot, getting that app, a simple chatbot app where you deploy the app yourself, that can be a really valuable experience. Something great to show off to recruiters, to somebody potentially interviewing you. And also, just pretty darn fun to build. You talked about database tools like Pinecone, software engineering skills that you can develop through tools like Lead Code online, SQL of course, which we already talked about earlier in this episode, and Pandas. It all makes sense to me. 

01:46:17

There aren’t any huge surprises in there, but it’s nice to have these kinds of things reinforced and especially these particular technologies like Lead Code, Pinecone, those are useful for our listeners, for sure, and I’ll be sure to include links to those in the show notes. So we only had one other audience question here that I don’t think we specifically addressed. But I think over the course of this whole episode, Hartron, I hope you feel like we have answered your question. So Hartron Kabe said he’d like to understand how the developments in generative AI like LLMs are shaping the job of data scientist. And so he asked, “Do you think data scientists might transition toward roles similar to AI/ML engineers? While others may specialize in areas such as data strategy, ethical AI or the interpretability of AI models?”

01:47:05

And I feel like based on everything you’ve said in this episode, Daliana, that it’s safe to say that yes, there’s going to be more and more specificity, as we have more and more of these kinds of AI tools, as we have more and more tools for abstracting away the complexity of those, we’re going to have more and more specializations develop where people will be ethical AI experts more and more, interpretable AI experts more and more. But I think to summarize a key point from this whole episode from you is that we also want to be mindful about what the day-to-day is really going to be like in a role, and not necessarily just focus on the title. 

Daliana Liu: 01:47:44

If one thing I could interrupt, I think you are right on, there are more topics, specialization people can get into. But right now my view is, I don’t think practically there are going to be those titles with those specializations. So I think those AI skills, specialization, is going to be weaved into the current role. I don’t think a company can afford to hire different roles. So maybe those AI skills are going to be expected in data science or ML engineer. I remember a couple of years ago when MLOps was really hot, and there would be MLOps engineer, but we don’t see that much anymore. It’s just software engineer working on ML platform, or a data scientist doing MLOps. Again, sometimes the company will create those job titles. I think the job title doesn’t matter. It matters less and less these days. When you look for jobs, it’s focusing on the skillset. 

Jon Krohn: 01:48:51

Nice. Yeah, very well said. And I agree a hundred percent. All right. So Daliana, you’ve been extremely generous with your time with me today. We’ve gone way over the scheduled slot that I had with you, so thank you very much for that. Before I let you go, do you have a book recommendation for us? 

Daliana Liu: 01:49:07

I will share the book that I read from my career coach. Her name is Victoria Song. I found her coaching program through this book called, Bending Reality. I highly recommend it. You can listen to the audiobook. She shared some tools and techniques for you to understand yourself more, process your emotions, so resolve those blocks so later you have the courage, the strategies to go for the things you want to go. So that’s a book that recently had a great impact on me. 

Jon Krohn: 01:49:48

Nice. Thanks for that, Daliana. So yeah, very last thing that I ask all of my guests before I let them go is, how we should follow you. We’ve gotten some sense already. The Data Scientist Show may be the Daliana Show in the future. And obviously LinkedIn. Actually, I was checking, it’s as of today of recording, 280,000 followers on LinkedIn. So that seems like a safe place to follow you. Where else? 

Daliana Liu: 01:50:16

So you can also go to my website, dalianaliu. com. You’ll find my course. You can join the wait list through Maven. If you’re not ready for the course, you can also join my newsletter. Yeah. Oh, I forgot to mention. I just created a discount code. If you find a course helpful for you, I am going to create a code. I think it’s superds. If you use the code to register the course, you’ll get 20% discount. 

Jon Krohn: 01:50:56

For the Maven course? 

Daliana Liu: 01:50:57

For the Maven course, only for the Super Data Science listeners. 

Jon Krohn: 01:51:00

For Super Data Science listeners. Awesome. Yeah, we’ll make sure that we get that in the show notes as well, so that people can take advantage of that generous discount for our listeners.

Daliana Liu: 01:51:10

Thank you, Jon. 

Jon Krohn: 01:51:11

Well, thank you so much for coming on the show. It’s been such a fun and informative time spending this time with you. And yeah, people will catch up with you again in the future on the program. 

Jon Krohn: 01:51:26

What a great episode with Daliana Liu. In it, Daliana filled us in on common unhelpful career mindsets, such as assuming restrictive rules that aren’t really there, being driven by wounds from our past, chasing a job based on its title, not on its day-to-day responsibilities, and not considering the commercial implications of a technical project. She also talked about how you can find your niche under the big data science umbrella, or even outside of it by being honest about what you really want, including whether it’s actually money, or the approval of a loved one. And this will potentially broaden or narrow the scope of opportunities available to you. She talked about how building influence, selling internally and creating public content can accelerate your technical career, and the must-know hard skills for data scientists today, which include SQL, pandas, vector database tools like Pinecone, software development skills, such as those assessed by leak code, and experience working with generative AI APIs like the OpenAI API.

01:52:19

As always, you can get all those show notes including the transcript for this episode, the video recording, any materials mentioned on the show, the URLs for Daliana’s social media profiles, as well as my own at www.superdatascience.com/803. Thanks of course to everyone on the Super Data Science podcast team, our podcast manager, Ivana Zibert, media editor Mario Pombo, operations manager Natalie Ziajski, researcher Serg Masis, writers Dr. Zara Karschay and Silvia Ogweng, and our founder Kirill Eremenko. 

01:52:46

Thanks to all of them for producing another fun and informative episode for us today for enabling that super team to create this free podcast for you. Consider checking out the show notes to see our sponsors’ links, and check out what they’re offering, because that really does help us make this show and support the team. If you yourself are interested in sponsoring an episode, you can get the details on how by making your way to jonkrohn.com/podcast. Otherwise, share this episode with people who could benefit from Daliana’s guidance. Subscribe if you’re not already a subscriber. Review the episode on your favorite podcasting platform or YouTube. But most importantly, just keep on tuning in. 

01:53:26

I’m so grateful to have you listening and hope I can continue to make episodes you’d love for years and years to come. Until next time, keep on rocking it out there, and I’m looking forward to enjoying another round of the Super Data Science podcast with you very soon.