SDS 353: How to Practice Human-Centric Data Science

Kirill Eremenko: Welcome to the SuperDataScience podcast. My name is Kirill Eremenko, Data Science Coach and Lifestyle Entrepreneur and each week we bring you inspiring people and ideas to help you build your successful career in data science. Thanks for being here today and now, let’s make the complex simple. 

Kirill Eremenko: This episode is brought to you by my very own book, Confident Data Skills. This is not your average data science book. This is a holistic view of data science with lots of practical applications. The whole five steps of the data science process are covered from asking the question to data preparation, to analysis, to visualizaton and presentation. Plus you get career tips ranging from how to approach interviewers, get mentors, and master soft skills in the workplace. 

Kirill Eremenko: This book contains over 18 case studies of real world applications of data science. It comes off algorithms such as random forest, k-nearest neighbors, naive bayes, logistic regression, k-means clustering, Thompson sampling, and more. However, the best part is yet to come. The best part is that this book has absolutely zero code. So how can a data science book have zero code? Well, easy. We focus on the intuition behind the data science algorithms so you actually understand them, so you feel them through. And the practical applications, you get plenty of case studies, plenty of examples of them being applied. The code is something that you can pick up very easily once you understand how these things work. The benefit of that is that you don’t have to sit in front of a computer to read this book. You can read this book on a train, on a plane, on a park bench, in your bed before going to sleep. It’s that simple, even though it covers very interesting and sometimes advanced topics at the same time. 

Kirill Eremenko: Check this out, I’m very proud to announce that with dozens of five-star reviews on Amazon and Goodreads, this book is even used at UCSD, University of California San Diego, to teach one of their data science courses. So if you pick up Confident Data Skills, you’ll be in good company. 

Kirill Eremenko: To sum up, if you’re looking for an exciting and thought provoking book on data science, you can get your copy of Confident Data Skills today on Amazon. It’s a purple book, it’s hard to miss, and once you get your copy on Amazon, make sure to head on over to www.confidentdataskills.com where you can redeem some additional bonuses and goodies just for buying the book. Make sure not to [inaudible 00:02:47] that step. It’s absolutely free. It’s included with your purchase of the book but you do need to let us know that you bought it. So once again, the book is called Confident Data Skills and the website is confidentdataskills.com. Thanks for checking it out and I’m sure you’ll enjoy. 

Kirill Eremenko: Welcome back to the SuperDataScience podcast, everybody. Super excited to have you back here on the show. Today, we’ve going to very cool and interesting. All of our episodes are very cool, but today’s a very interesting episode because approaching data science from a different perspective. To provide some context, let’s try to answer this question. What are the outcomes that we’re going for in this specific data science project? So you might be working on something and how often do you ask yourself, what are the outcomes we’re actually going for? Or for instance, this question. How are we going to measure success in this data science piece of work or project or analytics tool, decision support system that you’re building, model, insights, how are you going to measure success? 

Kirill Eremenko: So the thing is that very often, we get caught up in lots of different things that comprise data science, from thinking about AI and data science strategy to juggling around different components of IOT systems, to working with data preparation or building different models, gathering insights, creating business decision support systems, and so on. Visualizing our data, presenting on it. We get caught up in all these different things. But what we might get in the end is in the words of Brian T. O’Neill, a technically right result or insight, but effectively wrong. What does that mean? Well … and or actually, why does that happen? Well, that can happen because along the way, we hadn’t been thinking about the end user, about their experience, about putting them in the middle of everything. That’s where human-centered design thinking actually comes in. 

Kirill Eremenko: Brian T. O’Neill is an expert in the space of human-centered design, specifically for enabling decision making to be precise, human decision making, in data science. So he’s bringing in the field of human-centered design which exists in other areas of the world as well, he’s bringing it into, or he has been bringing it for many years now into the space of data science and enabling decision making. So basically thinking about your customers throughout the way. 

Kirill Eremenko: That is a very powerful tool. It’s a specific soft skill but it’s not just about presenting the insights. It’s about thinking about your user throughout the whole journey. In this podcast you’ll get a lot of tips on how to do that. So Brian T. O’Neill’s a consultant in that space, he’s been doing it for many years. In this podcast, we will learn how to ask the right questions to understand the business needs and what actually is desired from a certain piece of work that you’re doing, the seven steps that he performs when he goes into companies to do his consulting work, understanding outputs versus inputs and the consequences or the outcomes that come out of your outputs. 

Kirill Eremenko: So lots of interesting questions will be raised in this episode and I have a feeling if you apply the things that you learn here in your next data science project, you’ll see a different attitude from the people that you’re going to be presenting and delivering it to. They’ll have a much better experience and results from your project. 

Kirill Eremenko: So there we go, that’s what this episode is all about. As usual, all of the links to connect with Brian T. O’Neill will be mentioned at the end of this episode, but I want to mention one already now. In case you don’t get to the end of the episode, but you want to learn more, Brian has set up a special page for us, thank you so much Brian. It’s designingforanalytics.com/superdatascience. You can learn more about his work there if you’d like. 

Kirill Eremenko: On that note, let’s dive straight into it and let’s learn about human-centered design thinking in data science. Here we go. Without further ado, I bring to you Brian T. O’Neill, founder of designingforanalytics.com. 

Kirill Eremenko: Welcome back to the SuperDataScience podcast, everybody. Super excited to have you back here on the show and today’s guest is Brian T. O’Neill calling in from Boston. Brian, how are you going today? 

Brian T. O’Neill: I’m doing great. How’s it going? 

Kirill Eremenko: Very, very good. I’m super excited about today’s show because we’re going to be talking about some really cool things, but first of all, you are a man of many activities and things that you do. So in addition to data science, you play the drums, right? Percussionist? 

Brian T. O’Neill: That’s correct. I do do that. 

Kirill Eremenko: That’s fantastic. Where can our listeners hear some of your work, because … we’ve chatted a bit about it before the podcast, but you play jazz mostly, is that right? 

Brian T. O’Neill: Jazz, chamber music, orchestral music, Broadway shows, that type of thing. Little less on the rock pop music. Occasionally, some stuff like that, but yes, a lot of jazz, classical, world music.

Kirill Eremenko: It’s not just like a hobby because you know, I play around, or I enjoy dabbling on the piano but I know two pieces. You actually play professionally. You have two lives effectively. There and here. How do you combine that? 

Brian T. O’Neill: Oh yeah. Well that was my training. You know, my formal training was in music. So I have a degree in percussion studies. I work as a freelance musician around Boston doing as I was saying, a lot of classical work. I play with a lot of the Broadway theater shows that travel through town in the pit orchestras for those. Occasionally some star attraction work, video game orchestras will come to town and they pick up musicians. Then I run a group called Mr. Ho’s Orchestrotica, which sounds like it’s spelled, it spells like it sounds, just orchestrotica.com if people are interested in that, that’s more of what I call my startup. It’s like running your own a little business and promoting original music in the kind of chamber, jazz, and global jazz kind of space. 

Brian T. O’Neill: So yeah, so I do that and then I’ve been designing for the webs for since 1996, if you blur that out. But yeah, about 25 years doing design, started out as a web designer, gradually moved into the kind of the Boston startup dot com scene and then got into more enterprise stuff, fidelity investments and JP Morgan and working in some banking and larger enterprise kind of contexts. Then kind of just got into some very nerdy IT related software products. There’s actually a lot of enterprise B2B companies here in the Boston area. I had clients when I started, when I went freelance in like 2006, I started working for myself so I could kind of balance my two careers. I just had clients that kept kind of bringing me along to their next projects and they tended to be at very technical companies so products for other IT people, technical data products, this kind of thing. 

Brian T. O’Neill: Analytics kind of was simmering behind the scenes and all of these and so that just became a kind of a focal point for me. About four years ago, I decided to kind of specialize my consulting work in data products. My goal is really to help companies design innovative and engaging data products powered by data science and analytics and really to focus on that last mile, which is where humans interact with the stuff that all of this great smart people that are doing things with data and math. At the end of the day if there’s a human in the loop in your system and you’re not developing a fully automated solution, then they are a factor in the success of the work that gets done. Doing that well is a different skillset than the modeling piece and the training sets and getting the data cleaned up and all those other things. You can get it technically right and effectively wrong, so I want to make sure that you know my clients and the people that I train in my seminars in this are focused on that human last mile piece. Really understanding the problem space, understanding how humans are going to perceive the work that’s being done, how they’re going to understand the data. 

Brian T. O’Neill: The visualization is part of this. We typically jump to that when we talk about design, we think of data visualization. I tend to think that there’s a layer, there’s a perspective that’s a little bit higher elevation, which is in the design world we call this the user experience kind of layer which sits above the interface. Because you can actually, you can technically get the data visualization piece right, but if you don’t have the right data to begin with and you don’t understand the context of use, then it doesn’t matter if the visualization is the best way possible to do it, whatever the heck that means, it doesn’t matter. Right? Because if no one’s logging in to use the service and to make decisions, you know … 

Brian T. O’Neill: That’s really what a lot of our work is about, right? It’s really about decision support. So if we don’t create decision support with these models and analytics services, then we’re really not having an impact with it. We have to look beyond the ink and the data ink that’s on the page and think about workflows and how do people do their jobs and what are they concerned about with this technology, do they understand it? What’s the change management that may be required there? That how I see it and one of the concerns I have … and jump in if I’m just like babbling here, but it’s the- 

Kirill Eremenko: [inaudible 00:12:25] very interesting. 

Brian T. O’Neill: Yeah, the reason that … one of the reasons that I know this is failing, I keep hearing this repeated over and over, is that what’s sometimes called the operationalization of models in the non-software companies, they tend to talk about AI and predictive analytics in this context of improving an internal business as opposed to creating a software product that has data science and analytics behind it. Those are kind of two branches and they use words like operationalization and change management. 

Brian T. O’Neill: From a design perspective, from a designer’s lens on it, I don’t like that perspective because I feel like what that means is this team goes and does the technical part and they’re going to spit out a spreadsheet, a visualization, a Tableau thing, a field and the CRM. There’s some output and then it’s some other team’s job to go in and make the business use that stuff. This is where I think things can break down, right? Because you’ve got two teams trying to do the right thing but I think the perspective that’s more important is did we design the solution, the model, the thing, the software application, whatever the output medium is with the engagement model in mind from the start and look at that as part of the success of the overall data science work? 

Brian T. O’Neill: It’s not a second thing. It’s not something that you pass off to another group. It’s integral to the work. Think of it as integral, not a deliverable that you pass to someone else to go shove down people’s throats. That’s not how you build stuff people want to use. Instead, you get them involved from the start, right? If it’s the sales team or it’s the CMO or the marketing department or whatever, they should be involved from the beginning of the project so there’s no big giant reveal. Like all of a sudden, the CEO is like, “Today, we’re making a big change. We’re releasing a new model to do X,” and jaws are hitting the floor and people are like, “F that. I’m not using that. I’m going to, I’m still calling the same … I’ve got my sales prospects, right? I’m not calling this list of sales prospects you came up with in the dark with some magic AI stuff. I don’t know what that’s about, but I know who’s going to buy this week and I’m going to call those people. That’s my job as a salesperson.” 

Brian T. O’Neill: Well right there, there’s your six months of data science work down the toilet, right? Because this salesperson does not want to, they don’t know how you came up with this list of strange customers that they’ve never talked to, but you’re saying, “Oh they’re going to close next week. They’ll sign on the dotted line next week. And oh, by the way, here’s what you should charge them. Here’s the price quote that you should use,” and the salesperson is like, “How did you come up with this number? Where does this from, I have no idea what this is about.” Because they weren’t involved with the solutioning and the problem discovery and they weren’t … There was no research done. That’s where things can totally break down. 

Brian T. O’Neill: So the designer lens is know these people are integral and we have to factor them in from the start and we’re all going to have a better time doing this work together because I’m sure … [inaudible 00:15:33] I’m sure you’ve had this experience, it’s more fun to work on stuff people want to use, right? Not stuff like, it just like- 

Kirill Eremenko: For sure. 

Brian T. O’Neill: You’re hoping this reveal, it’s like where’s the smiles? And instead it’s like, kind of quiet in the room and people are like, “What does that mean? What do I do with that number?” We don’t want to have those kinds of experiences. We want to deliver, like, “Yes, yes. When can I get more of that? Oh, could you also show me this? Does the model factor in this thing? Oh it does? Oh that’s awesome. I hate doing that work in spreadsheet.” That’s the kind of stuff we want to hear at the end. 

Kirill Eremenko: Yeah, totally agree. It’s interesting because I was interviewing Stratos, one of our students, just yesterday on the podcast as well. He said that when he was applying for data science job last year, at the interview for the job that he actually ended up taking, one of the questions was related to exactly this about soft skills. How he would present the data and data science project, how he would talk to executives, how he would go about helping people understand what insights he’s communicating. So it’s exciting to see that companies are not only realizing this after the fact now, once the data science projects are starting to fail, they’re doing it preemptively. They’re hiring people who know what they’re doing in terms of this, what you call it, operationalization and soft skills that change management’s in. 

Kirill Eremenko: What I wanted to ask you, so walk us through the process. You’ve outlined how important it is, it’s totally a critical part. What’s the point of doing a project if nobody’s going to end up using it? But once you go into a company and they need your help with this operationalization or change management or soft skills and data science, what are your typical steps? How do you identify the problems? Are they at the start, in the middle, at the end of the data science project pipeline or work life cycle? Then once you’ve identified the problems, what do you do about them? 

Brian T. O’Neill: Well, the most popular response to this question by consultant ever is, “It depends.” There is a broad design process that I typically use, but that process is more like, it’s more like a shelf of ingredients. I may or may not use this ingredient with this particular pie this week with this client or I may use a ton of it. Or I may start with the flour and then add the water later and the next time, the flour doesn’t come in until way later in the process. 

Brian T. O’Neill: One of the things that clients need to understand when they’re doing this type of work, when you’re doing creative work, when you’re doing discovery work to get into people’s heads and understand the problem space and all of this, is that it’s not highly analytical work, ironically. You’re going to have to ping pong back and forth to understand what’s needed. So sometimes you actually, you need to get into the design itself in order to figure out what needs to be designed. 

Brian T. O’Neill: Research is a big thing that’s often missing in this place and that can sound like this really expensive long thing that takes forever to do. Nope. A lot of times what I’m talking about is having one-on-one conversations with the actual consumer of whatever it is that’s going to … whoever’s going to use this. You know, if we’re talking about predictive analytics, whoever’s going to use this predictive score, we need to figure out what is going to make them want or not want to use this from the beginning. 

Brian T. O’Neill: You may even need to start with, well, we don’t even know who’s going to use this, and so right there we haven’t even figured out who is our team, who are the stakeholders in this project, what are their interests in this, and what is going to make or break this? You may need to have a conversation with your senior level stakeholders, because sometimes what can happen is you can’t even get the time you’re trying to… Say you’re helping out the marketing department and they’re like, “We don’t have time to sit in your ideation sessions to go through this.” Well, senior management needs to hear that, and the response from… If the data science experts are the ones that are leading the process here, what they need to hear is, look, we can build a model for anything if we have the right data for it. 

Brian T. O’Neill: If you want us to build a decision support mechanism for the marketing department, the marketing people need to be involved in the process of helping us understand the pain and the need, and how they’re going to use this. If they’re not, what you’re going to do is pay my department $5 million over the next three months, and you’re going to have a high risk output at the end of that. So, do you want to take the chance that all the work we’re going to do is going to hit the floor and it’s never going to be used, or do you want to make sure that the marketing people are saying, “Wow, we know how to stop advertising in the wrong spot. We know who to send our mailers to for the next campaign that’s coming out. This is really helpful information.” 

Brian T. O’Neill: And if you want the ladder, you’ve got to have those people involved at the right time. So, there needs to be a clear understanding of who our users are, our end users, who our stakeholders are, how we’re going to understand what it means for this output. Again, our visualization, our predictive model, whatever it’s going to be, how will we measure the success of that at the end of the project before we get into building anything, right? And what we may find is that you don’t need a machine learning model for this product. Maybe the first version of something is like, you know what? Right now you’re taking a wild ass guess every time you decide which cohort of people are we going to send this campaign to, right? 

Brian T. O’Neill: Well what if we could just simply tell you how many people open the mail that we sent last year, and we can compare it to these other metrics. And it is, is just historical data, but we can at least get that going. Would that be an improvement? In one month, would that help you start making better decisions about where to spend marketing dollars? And if they said, “Yeah, that would actually be really great,” well now we have a way to start small, and we’re really focused on that business outcome, right? Instead of focusing on building a model, we’re focused on help the marketing department know who should we send mailers to and who should we not send mailers to for the Spring 20/21, whatever, shoe campaign, right? 

Brian T. O’Neill: So, 

Kirill Eremenko: Yeah. 

Brian T. O’Neill: So, that’s part of it. There are several different steps. In my seminar, I have these, approximately seven different steps. You have your team building, we have a stage of research and problem finding or problem definition, which is where we get really crystal clear with our team about what problem we’re trying to solve, not what data science problem we’re trying to solve, but what people problem are we trying to solve, and what business outcomes we’re going for. 

Brian T. O’Neill: We then move into starting a design brief, and this is where the question of ethics starts to come in. So, we may be looking at what are the second order consequences of the work that we’re doing here? Where might we need to put checks in place with the work that we’re doing so that the solutions we’re building are ethical, and useful, and usable, and actually consciously thinking about this, and not just waiting for a story to hit the news that we don’t want to hear about. So, that’s a factor in the process. From there, I’m a big fan of using a couple of tools from design, which are called journey maps or service blueprints. So, maybe you’ve heard about these before, but this is a visual way of talking through the customer’s journey, where they are today, and how they do their work today, and plotting that out visually over time so we can understand what’s it like to be this marketer that needs to send out things. 

Brian T. O’Neill: How do you decide how to do that today? What process do you go through? Well, we collect these analytics from this tool, and then we go into Tableau and then we look at, whatever, the CRM, and then I kind of take a guess based on what I think the market’s doing. Anyhow, we map this thing out, and by understanding this customer journey and all the departments that may be involved, we can start to have a bigger picture about how does our little, or maybe major data science initiative fit into that workflow, and where might it hit the ground, right? Where is there a gap that may not be a data gap, but it may be an engagement gap, like trust. 

Brian T. O’Neill: Maybe we find out the salespeople are on the road all the time. They’re not going to open up a PDF, they’re not going to open up Tableau in some desktop thing or whatever. They’re only going to respond to text messages, whatever’s on the screen in their little app that they use. We need to provide them with really good recommendations on which door should I go knock on next if I’m selling widgets door-to-door, or something like that. We need to understand what it’s like to be that person and to do that job, so that we have that context the entire time that we’re doing our work. So, the journey maps and service blueprints can help with that. The difference there is really whether you’re talking about external customers’ experience, or you’re talking about internally how a business process works. So, the service blueprint is really for if you’re building a model or something to improve operations inside a business, it’s… They talk about the front stage and the backstage. So that’s what that service blueprint version is. 

Brian T. O’Neill: But they’re very similar. They look very similar. And then from there- 

Kirill Eremenko: So, the service blueprint is internal? 

Brian T. O’Neill: Yes, it covers… think, again, think of it as the backstage, like the behind the scenes. When you go to the Apple store, it’s like thinking about all the process of how they onboard you when you come in and have a customer service on your iPhone, right? And then, all of a sudden they walk away with your cracked iPhone screen. Well, behind the scenes there’s a whole bunch of stuff happening there. They’re probably like, is it under warranty? No. Okay. If that’s the process, then we do a quick five minute check of the screen, and see if we can do a hot swap. Nope. Okay. There’s a whole process that they go through there, but the customer doesn’t see that, right? So, that’s more of that service blueprint version. 

Brian T. O’Neill: So that may be applicable for your audience that’s working as an employee inside a business, trying to improve the internals of that business from there, yeah, and another part of this is what I call the honeymooning and the onboarding. And this is what, again, we sometimes use this term operationalization, but I also like to think about what I call the honeymoon period, which is the period between when we make the announcement or we “launch,” launch or put into production, whatever the output of our analytics work is, there’s a period of time here where it’s new and it’s different, and I call this the honeymoon, and your design and the way this works, you may need to consciously put intentional effort into how we help with that transition, instead of relying heavily on training, which it’s hard to get people to show up for this stuff. It may be something where we actually need to design into the software. 

Brian T. O’Neill: For example, how do we transition someone from the old way? We know that you used to use a spreadsheet here. Well you can actually upload your spreadsheet here, and then we’ll map this into our predictions, and we’ll help you save some steps of maybe they need to key in a bunch of data in order to get back some recommendations on something, and by understanding what these blockers are, the friction points, we can actually smooth this transition out by really… And I’m sure you and your listeners have experienced clunky onboarding when you download a new app for your phone, and a lot of times they force you through a tour, and you’re like skip, skip, skip, skip, skip. Just get me into the product, right? 

Kirill Eremenko: Yeah.

Brian T. O’Neill: You kind of want the product to just be intuitive. You don’t want to read a bunch of screens about all this stuff it’s going to do, because you probably downloaded it because you have one thing you want to do, and now they want to tell you about 20 things that you need to do through a video or whatever. All that stuff is just in the way. But if you really understand how someone wants to use the service, and what their job is, or what they need to do, you can design that experience to gradually bring them into the new way of doing whatever that may be. So, from there you get into actually doing the sketching, algorithm design planning, getting visual with workflows. If there are visualizations that need to be presented here, then I like to work low-fidelity. 

Brian T. O’Neill: So, I kind of teach this idea of working lo-fi with a small team with your power team that we talked about in the first module there, but working at a whiteboard together, and trying to get visual, and to prototype or simulate what might our outputs look like in low-fidelity before we ever do any data work whatsoever. And again, partly what we’re doing here is we’re taking away the giant reveal. It shouldn’t be the… There’s a black cloak. You walk into this dark room and then, bang, the lights go on, and here’s the data science model. That is not how you want to release stuff. 

Kirill Eremenko: It’s kind of the difference between Waterfall and Agile, I guess. 

Brian T. O’Neill: Yeah, exactly. So, we want people nodding their heads as we move through this process because they know what we’re doing, they know why we’ve done it the way we’re doing it, and they’ve been involved throughout the process. So- 

Kirill Eremenko: And they feel like owners as well. 

Brian T. O’Neill: Yeah. 

Kirill Eremenko: In the end the product is you’re presenting it to their bosses, they’ll be on your side helping you present it rather than on the opposite side. 

Brian T. O’Neill: Exactly. Exactly. So yeah, so there’s this visual process there, and we’re doing this work and iteration with our team. And then, kind of the last two, really the last formal phase here. And again, remember by this point, you may realize, wow, we don’t even know what we’re trying to do here. We haven’t clarified the problem yet. We’re, we’re sketching stuff, but we’re realizing by getting visual here, our chief marketing officer has been on our team here participating, and they still don’t really understand how they’re going to use this number, that our model is going to come up with, to do their work. 

Brian T. O’Neill: We might need to go back to the drawing board, do some other research or talk about the problem space more before we go any further, before we start doing a ton of work, collecting data and building pipelines and all this stuff. You may ping pong back and forth between these different stages before you move forward. But the last kind of formal… If we were to do this in the perfect theoretical way where it was step one through six, perfectly, the last step here would be doing validation of the results here. 

Brian T. O’Neill: So, what does that mean in the context of a predictive model or something like this? Well, the easiest way to boil this down would be, let’s say you’re going to present a score from zero to 100. You have a probability that’s what your model spits out. And the CMOs going to… Every day they log into this dashboard and your model’s going to produce a 67. On Tuesday, let’s say the score is 67. Well, what are you going to do with that 67? And 67 as compared to what? And asking this person, well, what would you do with this 67, and letting them talk about how are you going to react to this score? 

Brian T. O’Neill: So, by presenting them a visual and having a conversation with them, we sometimes call this usability testing or design validation, we can start to tease out what might need to go into the engineering and the modeling. So, what you might hear is, “Well, 67 doesn’t feel very certain to me, but if I understood why it was 67, then I might know who to send my mailers to, right? But right now you just say it’s 67, and I don’t really know how you guys came up with that.” So, ding-ding, light goes on, right? We might need model interpretability here, right? We may need a way to show which features contributed most to that. And if they didn’t say that, and someone said, “You know what? Anything above an 80 I’m cool with that. I don’t really give a crap how you guys came up with it because it doesn’t matter. Anything above 80 is awesome. Anything below 50 I’m just going to totally ignore. I don’t care.” 

Brian T. O’Neill: Well, at that point you may say, “Well you know what? We can come up with a much better algorithm here that’s 92% accurate, if you don’t really need to know how we came up with this, and there’s no compliance issues,” or whatever, that can start to guide the technical decisions that are made in terms of how the actual data science part works. But guess what? You don’t need data to test this out. You may find out that, oh with the 67 we need this kind of model interpretability, and in fact maybe we need a scoring system, qualitative ranges like, anything between 67 and 75 is a buy. Anything that’s below 67 and 52 is a hold. Red, green, yellow. Sometimes we talk about the traffic lights. 

Brian T. O’Neill: The point here is you’re talking about putting a qualitative measure onto this quantitative score that came out, and the only way you’re going to know what those qualitative ranges should be is by having a conversation with your customers, with your users, to understand how they’re going to perceive this information and how they’re going to act on it. And you don’t actually have to build the entire model to know that. You can also start to tease out how accurate does it need to be, and this is something I hear about, a fair amount is data scientists, especially young ones, they want to do cool data science work. Some of the more academic ones want to publish papers about how accurate their models are, et cetera, and they’re focused heavily on the accuracy of the model, and not so much on the, “did someone use my model to make decisions?” 

Brian T. O’Neill: In a business context, that’s what they care about. And what you may find out, I have a podcast episode about this, the title was something along the lines of, “When does the 60% accurate model beat an 85% accurate model?” And the joke is, well it’s the one that actually gets used to make decisions. That’s what matters. And the reality was is this person was David Stevenson, he was talking about this was a big light bulb moment for him. It was when he learned from his client, I forget it was an employee, or if he was consulting there, but the client, he was spending all this time trying to get the accuracy up from 80 to 82% or 85%, and his business sponsor was like, “What the hell are you doing? This is so great. If you can tell me that this is 65% accurate, let’s go onto the next thing. I don’t care. I’ve made my decision. It’s a yes/no decision.” 

Brian T. O’Neill: I forget, I’m kind of paraphrasing it. This could be wrong, but the point here was that was more than enough accurate for some really great business value to be created. And spending twice as long to get a 5% increase in the quality of the prediction was not a good business decision whatsoever, because now you’re spending all your time doing this work that the sponsor or the user doesn’t care about. It won’t make any difference on how that person does their job, whether it’s 80% or 85% accurate, it’s completely meaningless. But if you never have those conversations with your stakeholders and your users, you’re never going to know that. You’re not going to know what the pain and gain looks like from their perspective. This empathy is what’s required for us to understand how to design really effective solutions. We have to put ourselves in their perspective and take off our technical hats, and look at things from the perspective of the person that’s consuming them. 

Brian T. O’Neill: That’s what design is really about. It’s really about empathy and being able to put ourselves in their seat and their role, and relate to what that person’s job is like, and how they make decisions about things, and how do we slide our technology in there too to help with that. 

Brian T. O’Neill: I know that was a long winded explanation of the process, but it’s mushy, it’s gray, it’s mushy, it’s not perfect, it’s not clean.It can be, we can put structure on it. That’s partly what we talk about in my seminar is, yes, it’s supposed to be a little bit messy. We may need to ping pong back and forth. That’s the innovation space. But we’re also trying to fail fast here. We’re trying to learn quickly what’s working and what’s not without spending a ton of time building the wrong stuff. So, when you get that question, what is our machine learning strategy? Right there, your alarm should be going off. This is a bad question. This question needs to be unpacked. And the reality is, is your business sponsor, if you’re not at a software company, your business sponsor probably doesn’t understand what’s possible. 

Brian T. O’Neill: So you may need to have a separate discussion about, well, what is AI? What is possible with these technologies? And realize together we need to have a better conversation about what business outcome we want. Yes, we will try to use machine learning if that’s the best thing possible for us. And if you really just want machine learning no matter what, then let’s talk about this in laboratory, what I call, lab mode. Let’s have a project where really what we’re doing here is we’re going to rehearsal and we’re practicing. We’re having a scrimmage. 

Brian T. O’Neill: And if that’s the point, let’s take a really tiny project. There’s no expectation of business value. We’re just here to exercise our abilities to see if, can we collect data? Can we put the training data together? Can we test it? Can we deploy it? If that’s really just to exercise our skillsets and perhaps to see where do we need more talent? Is it visualization? Is it data engineering? Fine. But the point is you have a clear plan and a clear conversation to set expectations that the goal of this project is a lab mode to work on these skills, to see if in the future we actually have the skills to put AI to good use in a business context and produce some value. 

Brian T. O’Neill: I don’t think that’s what’s happening. I think usually it’s like, go give us some cool (beep), excuse my French, go build some amazing thing with… hire some PhDs, and they’re going to come up with this magic sauce, and they’re going to give it to us at the end. And then, there’s this big disappointment, or the data scientists are saying, “Well, what is the problem you want me to work on?” 

Brian T. O’Neill: And the business person is saying, “Well, what’s possible?” And they’re like, “Well, I don’t know, you’re the product manager. What would you like us to help you with?” And you can see there’s like a tennis game going back and forth. And my feeling is, and the people that I talk to in my show, it’s time for the data people to step up here and start to have a better understanding of the people that are going to use these solutions. 

Brian T. O’Neill: And it’s not to say that business people don’t also have a responsibility to become more data literate, but I tend to think that the last straw in this game are the people that are writing the code and pushing this stuff out. It’s the data people. And so they are the linchpin in this. 

Brian T. O’Neill: And I think that skill set needs to be developed, at least in part, by the data people. They need to learn how to ask good probing questions. They need to learn how to extract these needs from a stakeholder who may not understand what’s possible yet with these techniques, and to try to really guide that person to express their need more clearly so that your team, if you’re the data people, you can be assured that my work is not going to fall on the floor in six months. People are not going to be wondering what is the value of paying… And this is expensive, right? People are paying top dollar right now for this talent. But guess what? It’s going to change. 

Brian T. O’Neill: The salaries are going to come down. Everyone’s jumping into this space and at some point there’s going to be a lot of people with “data science” in their title or claiming this, and who’s going to be left standing are the ones that can actually turn data science into value and outcomes. And that requires a different skillset. It’s not Python, it’s not R, it’s not Kubernetes, it’s not all that technical stuff. That’s part of it. But there’s another part of it here if you really want to connect it to the people. 

Brian T. O’Neill: So, anyhow. I’m blabbering. You asked me some questions, but I’m hoping this is helpful to your listeners. 

Kirill Eremenko: Very helpful. I’m listening, soaking it all in. Very interesting insights. 

Kirill Eremenko: What I think would be very helpful for our audience is the concepts you identified fantastic. From team-building to design brief, journey maps, service blueprint, honeymooning, sketching, algorithm design, validation, usability testing, very useful tips. 

Kirill Eremenko: However, it sounds like that is something more of, it’s good to be aware of for anybody, but also that’s a framework for a consultant who goes in to analyze a business or maybe for even a business leader or manager to work with their stakeholders around data products. 

Kirill Eremenko: The question that I’d love to get your opinion on is, what can an individual contributor, an IC data scientist, somebody who’s there building the code, who’s not the only data scientist in the whole company where he would obviously have the mandate to apply this framework of certain steps, but he’s a part of a bigger team. Maybe there’s a hundred people in this team, maybe there’s five or 20 people in his team. He’s not the manager, he’s not the leader or he or she. He or she, they are part of this bigger team so they don’t have this control or say over what’s going to happen, what’s not going to happen. They’re just doing their job. How can they be better at design thinking? 

Brian T. O’Neill: Sure. It’s a fair question. And ultimately these are strategic questions. They do come down to that. I think the way to think about this is by being objective and asking good questions. For example, how are we going to know that we did a good job with this project? If you’re having questions about the work you’re doing and you feel like, “God, this project is going off the rails,” well maybe it’s time to get your team together and just have an informal conversation and say, “It would really help me, could we come up with just five bullets that are going to dictate the success of this project?” 

Brian T. O’Neill: How would we know, not technically, not anything to do with the data, but at some point we’re going to present something to somebody, right? They’re going to consume this and they’re going to make a decision about whether it’s good, bad, okay, excellent, whatever. How was that going to happen? 

Brian T. O’Neill: And if there’s silence in the room or it’s really mushy, then you can express that. Should we spend some time clarifying this so that we can make sure that we really hit a home run here? To use a baseball analogy, are we hitting a home run or a base hit, here? Well, if no one can tell us the difference between what a base hit and a home run is, then what do you think the chance of us hitting a home run is? It’s probably pretty low. 

Brian T. O’Neill: It sounds really simple here, but if you have responses like, “We will impact the business, we will use AI in the CRM. Well, we could probably find an office shelf thing, create a field in the CRM and shove a data point into it and say, “We created AI in the CRM.” Whatever, right? 

Brian T. O’Neill: That’s not clear enough for us to actually be actionable here and produce value. And I think if you really ask these questions with your team, and it’s not meant to challenge anybody, it’s in pursuit of clarity for the team so that we don’t create a data output that falls on the floor. 

Brian T. O’Neill: And part of this is how you ask the questions. I have articles on my site about how you do this kind of research, but a lot of this really comes down to asking really good open-ended questions, listening and trying to kind of facilitate the conversation here so that everyone sees why we’re asking these questions. But I think that’s one way to do it, is simply to have a question about what are the outcomes we’re going for, here, and how will we measure that these were effective? It’s very rare, I hate to say it, it’s rare that I see that because most of the time employees are usually compensated for inputs and effort. You trade your time, we pay you a salary every month to come in and use your data science skills and you’re really paying for time. 

Brian T. O’Neill: This model focuses more, even though you’re still going to get paid for your time, this model comes from a different one, which is, “What if our compensation was based on the results that we created, the outcomes and the value that we produce?” And I think part of the reason we don’t have this question as much is partially because of the way most companies compensate their employees. And I’m not suggesting we change that or anything, but that’s, I think, part of the reason these questions don’t always come up. And instead we just kind of wait for a boss to tell us “This is the next thing we’re doing. Here’s the project, here’s where the data are, build some connectors. We’re going to need to clean up this X, Y, and Z, dah, dah, dah, dah.” And you kind of just go in and do the work. 

Brian T. O’Neill: At some point in your career, if you’re a junior and you’re probably going to need to go kind of down the expert/contributor path, or you’re going to go into management, but either one of those two things, the more you can start to realize that I’m not really being paid here for our programming, that’s not really what they want. The reason why someone is funding my team is they want the value that our programming theoretically can produce for the business. And if you don’t know what that value is, it’s going to be a lot harder for you to be seen as a great contributor. 

Brian T. O’Neill: And so trying to align your work with that bigger picture, that’s one way to really start to connect the dots, here. Is to say, “You know what? Yes, I know how to do the model. Yes, we have the data here. But my concern is when we talked to so-and-so, the CMO, they said, we’re not going to use this. If we can’t understand how you came with this prediction, we’re not going to be able to use it. We can’t take the risk, and we’re going to keep the status quo here.” 

Brian T. O’Neill: So when your boss is saying, “Look, we can get 95% accuracy by using this deep learning model, blah, blah, blah,” then you can say, “Well that’s fine. I’m with you if that’s what you want to do. But didn’t we hear so-and-so said they’re not going to use this? Do we want to maybe try a first version here, maybe little bit less accurate, but we know this person’s going to use it because we can prove to them how the model came up with these recommendations. Maybe that’s where we should go first and then we can see if we should make it more accurate.” You can try to have these conversations and I know that’s tough. 

Brian T. O’Neill: Sometimes it’s hard to have these conversations when you’re early in your career, but I would challenge your listeners, you’re not really there to write code. What really a line of business wants is they want the value that your code produces, but it’s not really the code and the modeling and all the stuff that you learned in school, that’s not really what they want. It’s the output. It’s the outcome from your outputs. 

Brian T. O’Neill: So if you always have that kind of lens in your mind and you can connect it to the people who are going to consume those outputs, you’re probably going to be more successful in your career in general. 

Kirill Eremenko: Wow. That’s golden. I think if people follow that advice, they’ll be twice as successful already. 

Brian T. O’Neill: Yeah. 

Kirill Eremenko: Indeed. 

Brian T. O’Neill: Sure. 

Kirill Eremenko: It’s the output. As you said, the outcome of the output.

Brian T. O’Neill: Yeah. 

Kirill Eremenko: That matters. 

Brian T. O’Neill: Can I give you an example of this? Like real quick. 

Kirill Eremenko: Sure. 

Brian T. O’Neill: This is my learning moment for your listeners. When I was working, I worked at Lycos. If you remember, Yahoo was big. 20 years ago, Yahoo was the big search engine before Google and Lycos and AltaVista were competitors. And I worked at Lycos and I was a designer and we each belonged to different verticals and I focused a lot on the financial services products and some online trading platforms and things like this. 

Brian T. O’Neill: I remember one day when I was designing one of the stock research pages for the Lycos Finance or whatever it was, and I was talking to the product manager about the ad placements here. And for years, and most designers, they hate the ads. If you’re working in a media company where advertising is the model, you have to find these slots to put banner ads on the page and you hate it and all this kind of stuff. 

Brian T. O’Neill: And it didn’t click until this moment, this conversation with him, that, wait a second, these ads are what fund my salary here to be a designer and to do the work that I love to do. So what if I change my perspective to, look, no one really likes looking at ads. We know the customers, they hate that, but at some point this funds the business. So what if I can use my creative energy to figure out how to fit adds into the experience in a way that’s not so annoying, maybe it’s a little bit smoother or maybe there’s an add at a surprising place where it actually has some interesting context for the customer. Something like that. 

Brian T. O’Neill: I looked at it more like I actually want to help my team produce more advertising revenue because that funds my salary, but that’s really what they’re asking for. And it was just this light that kind of went on for me. And so I stopped fighting it and I realized it’s never going to go away. This is a media company. At the time they were looking at subscription businesses and other models, but at the time it was a media company, which meant advertising. 

Brian T. O’Neill: And so I actually started coming up with some other advertising products that we could actually go out and sell, that kind of stayed out of the way of the UX, which was my job, is to make really great user experiences with these interfaces. But also, I’ve started to think about what would be some other ways we could sell creative advertising, because that’s really what it was about. 

Brian T. O’Neill: But I was fighting it constantly. And part of that, you want that ying and yang, right? You kind of want some of that in the business, which is you have kind of our purist designers. Designers can relate to this, they’re always going to go for simplicity and usability. And sometimes you may want to encourage people to opt into a form, right? To provide some more data. Well, maybe there’s a creative way to collect that data that is both transparent and ethical, but perhaps it’s fun. Maybe you turn it into a game instead of asking someone to fill out a survey. We put that into a game context, but we realize that we actually do need to collect this information. How can we do that in the interest of the business and the customer? 

Brian T. O’Neill: It’s not just about the user experience piece. It’s about the business value that we’re creating, too. That was kind of the moment when the light went on for me in the advertising context. But I’m sure your listeners can probably find a way that they can start to see, “Wow, we’re going to help salespeople know who to call instead of just opening the CRM and smile and dial.” Right? What if we could tell them, “Here are the next 20 people, based on all the data we have, we think these 20 people are most likely to sign on the dotted line within the next two months.

 

Brian T. O’Neill: Put yourself in their shoes. What is it like to be a salesperson? And when you start to realize that’s really what the business hired you and your team for, is to help these sales people know who to call so they spend less time calling the wrong people. That’s what you’re there for. Not Python. Not R. 

Kirill Eremenko: Gotcha. Yeah. Wow, okay. So in a nutshell, keep in mind what you’re there for, in terms of you’re there for the outputs and the outcomes that come from your outputs, not for your inputs. And also, empathy. I love that you mentioned that, really understanding and sitting down. This really helped me many times. Sitting down with the people that I’m creating a model for, or creating some analytics, or doing analytics for. Sitting down with them, even just living with them through their whole working day. Understanding what they experienced, what they feel throughout the day, really helps inform what I need to do. 

Brian T. O’Neill: One other comment on this is, I’m going to totally cast a generalization, there’s lots of different people out there, but I’m going to say generally speaking, people with STEM backgrounds tend to be a little bit more introverted. They may find some of these kinds of research discussions a little bit uncomfortable. And here’s the great thing about doing good research: your primary job is to listen, it’s not to talk. 

Brian T. O’Neill: So if you’re not comfortable doing this, really your job is to come up with some good questions, we call them open ended questions, which means questions that generally don’t start with the word “do,” because we don’t want questions to end with “yes” or “no.” We want to ask, “Tell me about X. Tell me about how you decide who to call when you’re on the hook for your sales numbers. How do you decide who should get what offer?” 

Brian T. O’Neill: Ask open-ended questions, here, and just listen. And this is a way to kind of get get comfortable with this process where you don’t feel the need to talk like I am right now. I’m babbling, but it’s really about listening, is really what it’s about. 

Kirill Eremenko: Gotcha. Totally agreed. Brian, you have quite a few things that you’re doing at the same time. Of course in addition to your music, but at the same time you do consulting for companies, you run seminars, you are about to launch a course, which is very exciting. You’ve got a podcast of your own. First of all, I want everybody to know that Brian’s podcast sounds amazing. It’s called Experiencing Data With Brian T. O’Neill. Check it out on iTunes. Congrats, Brian. You’ve done what, like a year now? Of podcast. 

Brian T. O’Neill: Yeah, right. Episode 34, I think, comes out tomorrow actually, based on what we’re recording this now. So it’s been good. Yeah. Every two weeks we drop. 

Kirill Eremenko: Fantastic, really cool. So the episode you were talking about earlier is Episode 24, How Empathy Can Reveal a 60% Accurate Data Science Solution. So everybody, you’re on this podcast, you’re listening to this, this means you love podcasts already. Check out Experiencing Data With Brian T. O’Neill. I think you’re going to love it. And Brian, I wanted to ask you, what do you teach? You’ve already shared quite a lot of things on the podcast today. What is it that you teach, any additional insights you can provide from the seminars that you run? What are the discussions around there? I’m just curious what other themes exist in this space of using human-centric design and data science? 

Brian T. O’Neill: Are you asking what’s in a seminar or a course? Is that what you’re asking? 

Kirill Eremenko: Yeah, typically. What’s the news online [crosstalk 00:00:54:30]? 

Brian T. O’Neill: Sure, sure. I’ll give you an idea of the self-guided video course, which I just put up. The curriculum, like that process we talked about, those six or seven steps that were there, that’s a video course. So what it is, is for each module of those six or seven, there’s I think seven, there’s a short video where I kind of talk about the key concepts in there. And then there’s a written module that goes with that. And it’s really focused on doing the work. It’s not a read a book and you kind of digest 10% of it and 90% goes out the door. 

Brian T. O’Neill: Most adults learn by doing. And so what I really tried to do with this course is provide actionable steps for each module. What do I literally go out and do if I’m doing this work in my own organization? What do I go and do to put this into action? 

Brian T. O’Neill: When you talk about coming up with your team, well what does that mean? Literally? So that’s what I have. The video is kind of an overview for the module and then there’s kind of step-by-step activities there. And then I link to examples when it’s relevant. I try to provide some examples there. And one of the ways it’s different is that the course is called Designing Human-Centric Data Products. It’s loosely based on, you’ve probably heard of the term “design thinking” before, but what I felt was missing was this lens on data products. 

Brian T. O’Neill: So each module specifically talks about what is different in the context of data products. When I’m working with AI or probabilistic types of software applications, what are some of the considerations that are different? This is still a very new space. But generally speaking, I would say right now, a good 70% of the process is the same and 70% of the meat-and-potatoes of doing good design work is the same. 

Brian T. O’Neill: Whether or not it’s a machine-learning model or just descriptive analytics or some other technique, most of it’s the same, but there are other considerations we need to add on when we’re talking about probabilistic models. And so each module has a specific call out about what are the considerations here if I’m building a predictive model or something like that. So that’s the course. 

Brian T. O’Neill: And then there’s an instructor-led online seminar version, which is the same modules. The only difference is I release two modules per week, and then we have a call together with a cohort of people that are in Slack. So this is the doing it with other people, which partially helps keep you on track and makes sure that you actually go and do the work. 

 Brian T. O’Neill: And some people like to work alone, other people want to kind of have a cohort of people to go through it with and hopefully they can learn from each other, and so we have live Q&As. On Mondays, we release the new modules and have a discussion about those, and then Fridays we do a check-in and it’s actually spread out over four weeks. It’s not four weeks of 40 hours a week. It’s very much designed on it’s you get in what you put … 

Brian T. O’Neill: What you put into it is what you get out of it, and the goal here is to give you time to actually go and do some of this work, and some of it takes time. It takes time to set up, like I want to go do a ride along interview with my salesperson. I don’t know how they do they work, but I understand that I need to go understand a day in the life of the salesperson before I do this. Well, it could take a few days to set that up, and so I intentionally spread that seminar out over four weeks, so that there’s time for people to put this into play and then get feedback from me on it, so that’s the seminar and that’s how the training works. 

Kirill Eremenko: That’s very cool. Can you give us an example of, like you said, you provide hands on exercises for people to do on their jobs already? Maybe can you give us an example of a simple hands on exercise somebody can go and do at their work already tomorrow, to actually experience that feeling? 

Brian T. O’Neill: Well, in terms of actually literally spelling out how to do it, I don’t know if I could shortly do that on the podcast, but if you’re talking about what are some of the types of activities, like this journey mapping and service blueprinting we talked about, that’s one of the things we talked about, so who’s involved? Who do I need to bring to a session to do that? How do I set it up? In this case I actually provide a graph, a visual template that you can use here to get going with this, but we talk about literally setting up the room and who needs to be involved here. What is the goal of doing a journey map and what do you do with the thing? Okay, I have the map. Now what? 

Brian T. O’Neill: That’s what we go through in the course, is literally what is this for? What is the value of it? What do I do with it in order to move forward, and when do I do this? When is it important for me to use this particular tool? Because again, you may not need this tool or it may be too late for this at the stage of the product that you’re in, so that’s another aspect of this, is you don’t have to do everything and every single module all the time on every project. What I want to do is give you seven kind of core areas, and this is not all of design. This is it’s just like algorithms, or different modeling methods.

Brian T. O’Neill: You don’t necessarily use all of them all the time on every project, but I wanted to give people seven kind of core areas that they can go deep on. Six months later you may say, “Oh, wow! I remember we did something with usability studies and we have a lot of screens to show people. I’m going to go dig out that module on testing that Brian had, and then I can use that on this project, because we’re doing lots of visuals,” or something like that, so … 

Kirill Eremenko: Okay. Got you. What I love about online teaching is success stories, where people have applied what I wanted to convey, and they’ve gotten a job or a promotion or some kind of success. Any cool success story you can share about somebody who wasn’t using design thinking, and then through listening to you on the podcast, or taking your course, or somehow interacting with your work they decided, “I’m going to try design thinking,” and that completely changed their career? Anything inspiring like that, that you can share? 

Brian T. O’Neill: I can’t. For the seminar and the course I can’t, because the course is just about to come out, and the seminar just is actually really new and we just, I’m actually hoping to put some testimonials up soon. I’m in the process of actually gathering some feedback from my first cohort of students that went through this, so hopefully you’ll be able to see some of those, the results from the seminar and the course on my website in the near future. 

Brian T. O’Neill: So I can’t specifically say there’s that, but one of the things I got from one of the people in the course, and so this person was actually, she’s at an AI consulting firm and she’s technically an account and project manager, but what she realized was a lot of the challenges they were having, they were working with a pharma company, was in this kind of user experience space, because the client kept saying, “Well, just build some stuff and we’ll figure out whether it’s good later,” and she could smell that, “Well this could be really risky for us, because they’re happy with the work as long as we’re doing stuff, as long as we’re building pipelines and showing data,” but they couldn’t give a clear expression of how is this information going to be used to make decisions? 

Brian T. O’Neill: And she wasn’t sure. She could see the team was struggling with this and they had had some difficult conversations with their client, and so by taking the course she said, “I feel a lot more armed now about what tool do I use in this toolbox based on what the current client situation is that we’re having? And now I feel a lot more armed. When it’s time to test this, I actually know how to test the results of our visualization or the application that we built. I know how to go do that now with them, so that they can see whether or not the work we did was useful or not, and then we can learn from that,” and before it was very much is the client say they’re happy or not? 

Brian T. O’Neill: Well, the client may look at something and say, “I’m happy. That looks really nice,” but if you didn’t actually talk to them about, well, does it work well? If no one can measure how it’s supposed to work, then you might be just happy with the way it looks on the surface, but underneath the covers it’s not actually producing that value. And so now she knows how to go and have that conversation with the client and use that toolkit, so that was one of the big things for me, was that she had gotten that out of it. 

Brian T. O’Neill: Another student was actually managing director at a big supply chain finance kind of related company, and he’s trying to figure out, “How do I bring some of our IP and our analysts’ work that we do on every single project? We have some IP here. We want to productize this into an application, so that we’re not spending as much time doing the same types of manual tooling work here,” and he just didn’t have a framework for how do you get from ad hoc presentations of work to a software application that will express this information on a routine way? He was really struggling with that, and so he feels that now he has a much better idea of what that process looks like to get to that UI, that will help him spend more time doing higher value work for his clients. 

Kirill Eremenko: Amazing. That’s really, really cool examples of this stuff in action, and from quite some senior people as well, so you should give out certificates for your trainings, because this would be so valuable. It is already so valuable, but imagine somebody, I can just see somebody coming to an interview and saying, “Okay, this is a data science interview. What else do you know?” 

Kirill Eremenko: Python R, and normally people are like, “I know SQL, I know this. I know Tableau. I know Kubernetes. I know TensorFlow 2.0,” whatever else, and that’s all great, but imagine saying two, three, four, five, whatever tools, technical tools, and then in addition saying, “Plus I did a whole training on design centered thinking in the space of analytics, data science and AI, and this is what I know. This is the framework I apply. This is my awareness of the situation. This is how confident I am about dealing with internal and external stakeholders,” and boom! 

Kirill Eremenko: You just blow them away. Nobody ever says that at interviews, if you’re looking for a job or if you’re looking for getting better at your company, getting promotion or growing your existing business. It’s your annual review, whatever it is, or talking to your manager at your next one to one you. You start mentioning these things, discussing these things. You’ll be the first person in the whole AI team talking about design centered thinking. 

Kirill Eremenko: I think this is a great addition to anybody’s career, very exciting. I’m glad, very, very cool that you decided to go from music into this, and also, in addition, you’re finding time to teach this and spread this knowledge. I think that’s very, very, very cool. And also, plus this podcast, some interesting guests that you’re interviewing. Hats off to you for the amazing contributions you’re making to this- 

Brian T. O’Neill: Great. I appreciate the work. As for the certificate thing, someone asked me about this and I told her, “Look, this is the real world. It’s not school, and school’s about getting a grade,” and if you look at why is school the way it is? School was designed as, from what I understand, listening to a podcast on this, school was designed to optimize factory workers basically, right? You process them, you train them. If they don’t pass, you send them back, until they’ve learned the skill, and you move them up the ladder and you use quantitative testing to figure out whether or not they pass or not. 

Brian T. O’Neill: That is not the world of business. That’s not what we’re there for. It’s not to say I know more Python functions by memory than you do. That’s easy stuff to measure, right? The biggest reward I think you could get out of taking my course or seminar and the thing that would be the biggest thanks for me is when your profile, when your résumé starts to talk about the results of the work that you have created with your team, right? When it says I helped the business save $2 million a month by building a model that did X, that is going to make you stand out at your next job. When you say, “I know R Python, Kubernetes. I’ve certified Microsoft Cloud, whatever, blah-blah-blah.” 

Brian T. O’Neill: Well, guess what? You look like all the other people that are now coming into this field, except your number, I have seven years of this instead of six. Well, if you want to command a higher salary, when you can say, “Look, yeah, I don’t know Python as well as this other guy that you do, but did they help create a $2 million savings with their data science work? Do they know how to go talk to a stakeholder who says, ‘Give me AI,’ and they have no idea what they want? I actually can go in there and help you figure out what do they mean when they say they want AI, what should we really spend our precious data science dollars on?” 

Brian T. O’Neill: When you could have that conversation, and I think my course will help people learn how to do that, that’s a good thank you to me and that’s going to speak way louder than any certificate with my logo on it. I appreciate the gesture there and I know where you’re going with it, but that’s really what’s going to make a bigger difference for you as a data science practitioner, is to be able to really show the results and the outcomes that you have helped produce, and it’s hard to do this. Sometimes it’s hard to measure it and really track it back to exactly your work. I get it, but at least have that in mind with the work you’re doing and you’ll probably see you’re going to have a great career ahead of you I think. 

Kirill Eremenko: Fantastic, Brian. Well, that’s a great note to end this episode on. I think everybody’s gotten what they needed out of this and much, much more. Before I let you go, what’s the best place to find you? Where can our listeners contact you or get in touch, or just follow your career and follow the things that you share? 

Brian T. O’Neill: Sure. Yeah. I’d say if you go to my website, designingforanalytics.com, that’s probably the best place. I have a mailing list, so if you want to just keep track of what I’m doing, I do send out little insight articles every week and updates on the podcast, so each time we release an episode that gets sent out to the list. I’m also pretty active on LinkedIn. Twitter, my handle is rhythm spice, R-H-Y-T-H-M spice. 

Brian T. O’Neill: I’m not super active there, so I’d say hop on the list if you’re interested in following my work, and usually we have little, I offer little deals sometimes, especially when I’m putting out a new offering, a new training thing or something like that. I usually, in the spirit of doing MVPs, right? I may be contacting you to do a little research and like, “Is this a useful service?” And then offering coupons and discounts and things like that to my subscribers, so yeah, that’s probably the best place. 

Kirill Eremenko: Fantastic. Thank you, so once again, the website is designingforanalytics, all one word, .com. Definitely check it out, and the podcast is called Experiencing Data with Brian T. O’Neill. Brian, is LinkedIn a good place to, for people to connect? 

Brian T. O’Neill: Yes, that’s a great place to connect as well. 

Kirill Eremenko: Awesome. Great, and connect, make sure to connect with Brian on LinkedIn. Okay. Great, and one more question I have for you today is what’s a book you can recommend to our listeners? I’m sure you have something special prepared. 

Brian T. O’Neill: Yes. Well, there’s two books. I would say they’re both a little bit more on the business side, but are you familiar with Karim Lakhani, at- 

Kirill Eremenko: No. 

Brian T. O’Neill: He’s at Harvard Business School and he just wrote a text called Competing in the Age of AI, so I actually went to the little, the book release. I live close to Harvard here and I’m really just about 20% of the way through that, but if you want to look at, if you want to start to understand how is AI really changing the business landscape? And maybe you can start to feel like, “Oh, I can see how I fit into this,” I think that’s a good text to start to look at a high level, how your business stakeholders are looking at it. It’s not a technical book whatsoever. 

Brian T. O’Neill: The other text that I’m in the middle of reading that I’m enjoying is called Infonomics. I don’t know if you know Doug Laney, but this is a Gartner book and it really talks about how to monetize, manage and measure information, so it’s looking at data as an asset instead of this kind of like sawdust, right? It’s not sawdust. That dust has a lot of value, and so what do we do with it though, right? How do we create products with it? How do we improve products and services using data? And so Doug’s got a book there and I’m really interested in finishing that, so those are the two things that I’m reading right now. 

Kirill Eremenko: Fantastic. Thanks. Thanks for your recommendation, so Competing in the Age of AI and Infonomics. 

Brian T. O’Neill: Yes. 

Kirill Eremenko: On that note, Brian, thanks so much for coming on the show. Really enjoyed our chat and I learned a ton from here, and I’m sure our listeners will pick up great things from here as well. Thank you so much. 

Brian T. O’Neill: Awesome. Yeah, it’s been a pleasure to chat with you. 

Kirill Eremenko: So there you have it, everybody. That was Brian T. O’Neill and human-centered design thinking for enabling decision making in data science. How exciting was that? Lots of valuable insights, which you can already apply in your career already. Now, what was my favorite part? 

Kirill Eremenko: My favorite part was the concept of thinking about your end user throughout the whole process, so something that I’ve talked about before quite a lot is that the most in demand data scientists are the ones that can connect insights to end users, so that’s the last stage of the data science project lifecycle, the presentation, the visualization, the presentation, communication of insight, but Brian takes it a step further. 

Kirill Eremenko: He says that you need to be thinking about your end user not in the very end of your project, which is important, which indeed already sets your part, but he’s saying think about your user throughout your whole project. From the moment you ask the questions, to then preparing your data, to then building your model, to then visualizing and presenting it, the whole five steps you need to be thinking about your user. That’s what human-centered design thinking is all about in data science. It doesn’t matter if you’re creating a data science product, or you’re building a model, or just delivering an insight, or a decision support application, whatever it is, think about the end user.

Kirill Eremenko: I think it’s a skill. It’s a skill, it’s an art, something that needs to be learned and practiced, and hopefully now after this podcast everybody will be a little bit more inspired to practice it. And as mentioned throughout the podcast, you can find Brian at designingforanalytics.com, so if you’re a business and you want to engage Brian to look at your analytics products, then head on over to designingforanalytics.com/superdatascience. That’s a way to get in touch with him, and we don’t have any affiliate arrangement with him. That’s just a nice link that he set up for our listeners. 

Kirill Eremenko: On the other hand, if you are an individual contributor in the space of data science, if you’re a user or if you’re a data scientist basically, the best way, the best things that will benefit you from his suite of products or things that you can find on his website are his podcast, which is called Experiencing Data with Brian T. O’Neill, and since you’re listening to this podcast you already like podcasts, check that out. It’s Experiencing Data with Brian T. O’Neill. Then his seminar, which is an online seminar, and his course, which he just published recently or is publishing in the coming days. Check that out as well, so maybe you want to learn more about design thinking in data science, so there we go. That’s where you can find Brian T. O’Neill. 

Kirill Eremenko: Of course, you can connect with him on LinkedIn as well, and all of these links plus all the materials that we mentioned throughout this podcast will be available as usual at www.superdatascience.com/353. That’s www.superdatascience.com/353, and another exciting piece of news is Brian T. O’Neill’s coming to DataScienceGO, so if you haven’t booked your tickets yet, this is the DataScienceGO US version, United States, in October 2020. 

Kirill Eremenko: Brian T. O’Neill will be presenting there. We’re getting him to come all the way from Boston, so if you haven’t gotten your tickets yet head on over to datasciencego.com, get your tickets today, lock them in and we will see you there. You’ll see Brian T. O’Neill and lots of other exciting speakers, so there we go. That’s the end of today’s podcast. Thank you so much for being here today and I look forward to seeing you back here next time, and until then, happy analyzing.