SDS 883: Blackwell GPUs Are Now Available at Your Desk, with Sama Bali and Logan Lawler

00:00:54
Welcome back to the SuperDataScience Podcast. Today we’ve got not one, but two exceptional and complementary guests on the show. Sama Bali is an AI solutions leader at NVIDIA that specializes in bringing AI products to market. Prior to NVIDIA, she held a machine learning solutions role at AWS. She’s focused on educating data scientists and developers on AI innovations and implementing them effectively in enterprises. She holds a master’s in engineering management from San Jose State.

00:01:22
Logan Lawler leads Dell Pro Max AI Solutions. If you haven’t heard of Pro Max before, we’ll cover that in this episode. Over his 16-year tenure at Dell, Logan has held positions across merchandising services, marketing, and e-commerce. He holds an MBA in management from Texas State.

00:01:39
Today’s episode will be particularly appealing to hands-on data science, machine learning, and AI practitioners, but it isn’t especially technical and so can be enjoyed by anyone. In today’s episode, Sama and Logan detail why data scientists are camping out at 6:00 AM to attend NVIDIA’s GTC conference. They talk about the killer specs of NVIDIA’s next-generation Blackwell GPUs, how Dell and NVIDIA have joined forces to bring server-level AI power right to your desktop and how microservices are revolutionizing AI development and deployment.

00:02:10
All right, you ready for this excellent episode? Let’s go.

00:02:13
Welcome to the SuperDataScience Podcast. It’s awesome to have two guests, not just one on the show today. Logan Lawler, where are you calling in from today?

Logan Lawler: 00:02:29
Hey, Jon, thanks for having me on. So I am calling in from, well, I’m going to say Austin, Texas. It’s technically Round Rock, Texas, Dell’s corporate headquarters.

Jon Krohn: 00:02:37
Very nice. Then we also have Sama Bali on the show. Sama, where are you calling it from?

Sama Bali: 00:02:42
Hi, Jon. I’m calling from San Francisco Bay Area. Unlike Logan, I’m actually at my home.

Jon Krohn: 00:02:48
Nice. For our YouTube viewers, they can enjoy Sama’s beautiful background. It’s outstanding. It’s 99th percentile of guests we have on the podcast in terms of it’s very peaceful, but it’s also a bit officey. Just really nice colors. I love it.

Sama Bali: 00:03:07
It’s NVIDIA green.

Logan Lawler: 00:03:09
It is NVIDIA green. You don’t like the carpet color back there? The carpet color? It’s pretty good.

Jon Krohn: 00:03:14
Logan’s in a cubicle. It looks very corporate.

Logan Lawler: 00:03:16
It is.

Jon Krohn: 00:03:18
It looks like he’s in Austin, Texas in a cubicle.

Logan Lawler: 00:03:20
Yeah, exactly.

Jon Krohn: 00:03:21
He is.

Logan Lawler: 00:03:21
I am.

Jon Krohn: 00:03:24
Do you have any fun co-workers around you that you have Nerf gun things or whatever shooting at them in their cubicles?

Logan Lawler: 00:03:29
Honestly, part of my earlier career, there was a lot of that at Dell. There was a lot of hijinks and shenanigans. I would say as I’ve progressed in my career and got a little older, those hijinks have went away, which has hurt my heart a little bit because I quite enjoyed it.

Jon Krohn: 00:03:44
That’s too bad. I remember the last time I worked in an office with cubicles some people had remote operated, it wasn’t Nerf brand, but it was that same kind of idea of firing a completely harmless dart with a suction cup. They had turrets that you could remote control that were positioned on top of their cubicles. I do miss that. That is something I miss about that corporate setup.

00:04:09
Anyway, we’re not here to talk about desk arrangements. We are here to talk about amazing innovations. So at the time of recording this episode, we were just a week out from NVIDIA GTC, which is one of the biggest tech conferences in the world. I don’t know if you guys have some stats, maybe it is literally the biggest. So tell us about that experience.

00:04:30
Maybe we’ll start with Sama since you are formally at NVIDIA. So tell us about GTC. What’s that like for… I haven’t actually been to GTC myself and I bet a lot of listeners haven’t either. What is that like? We see so much of it on social media and the news. What’s it like in person?

Sama Bali: 00:04:47
The way it was described, it was described as the Super Bowl of AI and I don’t think anybody could have described it any better. San Jose is my college town. So I was really excited to see that we really painted the entire town green with all the innovations, not just from NVIDIA, but also our partners. Everyone from cloud partners to enterprises to startups. It’s amazing to see.

00:05:10
We are in the center of Silicon Valley, but to see that kind of innovation come to life was a great experience for me. I will say that. You could see all kinds of developers filled with data scientists, practitioners. Tons of opportunities to really network with people working in all kinds of tech companies at this point in time. Some amazing, amazing… I mean, I never thought I’d be having conversations with so many robots in my life in just the span of five days, but that happened.

00:05:44
So my favorite moment, although we’ll be seeing Jensen who is our CEO at NVIDIA, within the Denny’s food truck. I never in my world imagined that there is a food truck from Denny’s as well. There was one, he was serving food right before he went and actually presented the keynote. So that was my favorite moment for sure.

00:06:09
I’m also a nerd in a sort where I’ve seen people camp out, Apple offices whenever they’re announcing the new iPhone. For the first time in my life, I’ve seen people camp out to get into a keynote venue as well. People were lined up at 6:00 AM which was again insane in my mind. So it definitely was a festival in San Jose, I will say.

Jon Krohn: 00:06:30
Very nice. Logan, any highlights for you?

Logan Lawler: 00:06:33
I think Sama more or less covered it. I mean, it was my first GTC that I’ve ever been to and it was a lot. When I say this, this is meant to be a positive, but in terms of the learning, the education, just the booths set up, all the options, it was a bit overwhelming. I think the key thing is to go in with a plan of here’s what I want to accomplish.

00:06:57
Here is, for example, the learning track that I want to go down, whether I’m a data scientist or whatever, is to have that plan going in. It’s very easy to go stand in the middle of the trade show floor and just look around and just be in wonder for hours. I loved it. It was great. Tons of traffic, very packed, wall to wall. It’s not necessarily my jam, but hey, we were there.

Jon Krohn: 00:07:19
Fantastic. That sounds like an amazing experience. I’ll have to check it out in a future year. Maybe someone from NVIDIA or Dell will invite me and maybe I can contribute in some way actually in the future. I don’t know. I hadn’t thought of that before, but that could be a lot of fun.

00:07:34
All right, so particular to what both of you do, Dell unveiled at GTC last week, the Pro Max PCs. So what are those? Maybe that’s just, I don’t need to have an answer. What are Pro Max PCs? That’s a good question.

Logan Lawler: 00:07:50
What are Pro Max PCs? Okay. We, when I say we, I mean Dell, we kind of pre-launched, I wouldn’t say launched, but we announced these at CES. So overall, I’ll give you a quick bit of background, is Dell coming this year for many years, had lots of disparate brands. From professional to personal use, Precision, OptiPlex, XPS, the list goes on and on.

00:08:16
So really the rebrand starts there is how do our customers shop? That’s really broken down into what we’ve launched with a new brand nomenclature, which is Dell, which is for home and basic work. You have Dell Pro, think of your traditional consumer or your traditional professional, knowledge worker workflow thing like that. Then Dell Pro Max, which is what I support, is really designed for heavy ISV type workloads. Think like…

Jon Krohn: 00:08:40
ISV?

Logan Lawler: 00:08:40
Yeah, ISV, independent software vendors. So think like CATIA or Dassault software or Adobe, where it’s designed for specific workflows within industries that really rely on heavy GPU compute and acceleration to get their workflows done. So we announced them at CES, we launched them at GTC, which is to be honest, more of a kind of an ISG type show.

00:09:07
I was really excited with the amount of press, but also the feedback. We’ve been in the supporting data scientists, we’ve been supporting, that was our Precision line before. With the launch of Dell Pro Max, I think the key thing to take away is we still have our traditional, our towers, our mobiles. That hasn’t changed, all accelerated by Blackwell GPUs.

00:09:30
The difference is we did, and I know we’re going to talk about this so I won’t jump the shark too much, but we launched two systems that are specifically designed for data scientists and developers with system on a chip with Grace Blackwell designs, which really was a difference. I think a recognition from Dell and the market to say, we know where the market’s going.

00:09:49
We need to have a purpose-built device for data scientists that is easy turnkey, that brings the power of a server to the desk site. Which I know sounds crazy to say, but that’s really what’s happening. So that was the big announcement, which we can get into more later too.

Jon Krohn: 00:10:03
Very cool. So I guess the reason why that is announced at GTC is because of the inclusion of NVIDIA GPUs in the Pro Max PC line.

Logan Lawler: 00:10:13
Yeah, I mean absolutely. I mean that is correct. I mean we’re great partners. I mean that’s from an, I won’t say inception of AI, but over the last several years, the Dell AI factor with NVIDIA has been a cornerstone to our go-to market and how we make AI real. So it made perfect sense because workstations are very GP dependent. It’s the only thing that has the ProViz cards in the lineup. So it makes perfect sense to launch it at GTC.

Jon Krohn: 00:10:39
Cool. So, Sama, then, what’s your involvement from the NVIDIA side with these Pro Max PCs?

Sama Bali: 00:10:45
So I at NVIDIA lead our AI solutions go-to-market. So my job is to see when we are taking these new GPUs to market with our partners like Dell, how does that full solution come together? It becomes a solution once you really add in that layer of NVIDIA AI software to it. That’s what really truly becomes NVIDIA, Dell AI factory with NVIDIA.

00:11:07
We’ve got that entire hardware lineup from Dell, but then along with that NVIDIA AI software, which is consistent, you’ve got consistent experience, if you’re starting from workstations, moving to server, moving it to your data center, moving it back to your workstation as well. It’s that software layer which really helps you, one, harness the power of GPUs because this entire software layer is really optimized.

00:11:29
So your data scientists, your developers don’t really have to do that fine-tuning between the software or the AI models they’re using and the hardware at that point in time. So that really is my job, is to really talk about and manage the go-to-market for that entire full solution of Dell Pro Max PCs along with our NVIDIA AI software.

Jon Krohn: 00:11:48
Fantastic. So we will get to that software aspect in a moment. First, Logan, I think is the person to direct this at mostly. Let’s spend a bit of time talking about the hardware. So explain for my listeners what the latest in… Actually, I’m not certain that this is maybe for Logan necessarily, but explain what’s up with the latest in GPUs from NVIDIA as well as the different kinds of chips that are required to make something like a Pro Max PC be such a success for AI developers.

Logan Lawler: 00:12:20
Yeah. So I’ll take part of it. I mean, I know Sama will probably provide a lot more detail than I will and probably better detail. I think when it comes to Dell Pro Max acceleration by Blackwell, the card is at the core of it. Really any data science workflow, the acceleration, depending on what library is, Pandas or Polars or whatever, it’s really about how much can you load on the GPU, how quickly does that GPU work, and ultimately how quickly can you run that workflow.

00:12:45
Now there’s some surrounding things around that. I mean, GPU is core to it, but there’s other things within Dell Pro Max that also add to the experience and add to the ability to accelerate a workflow. I mean, our T2 Tower, which is even though it was announced last week, it launched today. I mean people probably won’t find this very interesting, but the size of that tower increased a little bit with not necessarily increasing the footprint that much.

00:13:09
So you can add in, for example, more hard drives for data storage, you can add in extra cards if you want to run multiple monitors. Things like that. Doing a few other things, being able to run on different applications, whether it’s data science or not, being able to run the new Intel processors at a 250 watt sustained workload, which is very unique in the industry.

00:13:29
That really wasn’t the case before. So all this has really been personally built. When you take the word max for Dell Pro Max, it means taking everything legitimately to the max. How far can we push this thing? That’s really Dell Pro Max in a nutshell.

Jon Krohn: 00:13:46
Very cool. So let’s talk a bit more about the GPUs in particular. I get the importance there, of course, of having CPUs. It sounds like ones there that are taking more power than ever before. So you have power requirements in this Pro Max line. Let’s talk about the GPUs specifically. What is the difference between Blackwell and predecessor GPUs? Why is having a Blackwell GPU from NVIDIA more helpful to someone who’s training or deploying AI models than GPU predecessors?

Sama Bali: 00:14:20
I can start, Logan. I’m pretty sure I’ll miss a bunch of things.

Logan Lawler: 00:14:24
You’ll be fine. You’ll do great. You’re great, Sama. Go ahead and then I’ll add on. I’ll add on.

Sama Bali: 00:14:29
Thank you. Perfect. So I think one of the things when we were talking to a lot of our data scientists in-house within our customer base as well, we soon realized in the last few years there was a lot of development, AI development happening in the cloud in the data center.

00:14:49
AI is now becoming mainstream. Everybody is trying to now fine-tune a model. Logan and I are not technical people. We are in the product marketing or guess what, we are fine-tuning our own models nowadays. We are running these models locally as well.

00:15:02
We soon realized that the systems we are using, you still need a little bit more horsepower in your system where you have the ability to actually have these data sets running locally. Have these models running locally, have the ability to fine tune, run, maybe a small RAG model just for yourself and things like that.

00:15:20
I remember talking to a data scientist last year during summer. He was talking about any time that he is trying to do a job, they had a Slack channel, he had to put his name down and he had to wait in line to get an instance where he could actually run his data set. So we recognize the need where now a lot of enterprise customers are struggling to get a lot more horsepower, and it’s not possible to give away cloud and data center just for learning, just for experimentation.

00:15:50
It’s interesting how AI is still evolving. Almost every day there is a new model you want to try. There is a new technique you want to try. So you need to have that local sandbox experience where you can just do your learning, your experimentation. If I am a developer who’s building an AI-based application, I probably want to continue doing all my testing because getting a data center resource is becoming more and more scarce.

00:16:15
So that was a thought process with the NVIDIA RTX Pro Blackwell GPUs. Then we’ve got a full lineup in desktop and laptop format for you. The biggest feature for me has been that we’ve doubled the GPU memory. So we’ve gone from 48 gigs to 96 gigs per GPU. You can actually have four of those in one workstation as well.

00:16:40
So that’s a lot of memory right at your desktop for you doing any running, any kind of model locally, fine-tuning that model. Any inference application that you want to run, you’ve got a lot of power right there within those GPUs itself, along with our usual more and more of our advancements we make with the GPU. That massive memory size to actually run these things locally has been a game changer.

Jon Krohn: 00:17:05
Excited to announce, my friends, that the 10th annual ODSC East, the one conference you don’t want to miss in 2025, is returning to Boston from May 13-15! And I’ll be there leading a four-hour, hands-on workshop on designing and deploying AI Agents in Python. ODSC East is three days packed with hands-on sessions and deep dives into cutting-edge AI topics, all taught by world-class AI experts. Plus, there are many great networking opportunities. ODSC East is seriously my favorite conference in the world. No matter your skill level, ODSC East will help you gain the AI expertise to take your career to the next level. Don’t miss – Online Special Discount ends soon! Learn more at odsc.com/boston.

00:17:52
So for 96 gig memory. So years ago, I’m kind of dating myself, the last time I built a server, I was buying 1080tis. NVIDIA 1080ti GPUs, which were at that time impossible to get. Everyone was using them for Bitcoin mining.

00:18:08
So I’d have to take an Uber, I was living in New York, I’d have to take an Uber to some distant Brooklyn warehouse to get one NVIDIA 1080ti GPU. They’d be like, that’s maximum one per customer. Then I’d have to try to source one somewhere else so that I could have two in my server. Those had 13 gigs of ram. So I could get 26 gigs into this server that I’d built.

00:18:38
For a while actually, with the size of models, even in early large language model era, myself as well as a team of three data scientists, all four of us were able to share two… I built two of those servers in the end and that was sufficient for us. It would compare to trying to run something on our laptop with just CPUs. It was crazy, crazy. You’re talking about 10,000 X kind of speed up to be doing that.

00:19:10
So it’s interesting now that the paradigm that you’re describing is being reversed because now somebody locally can have their own. In that kind of system that I was just describing there, we were a small AI company with a relatively small data science team, but there was no option for us to be buying individual machines that we could be fine-tuning our own models.

00:19:33
So we shared a couple of servers and because large language models weren’t absolutely massive like they are now, that was sufficient. As you’re describing, as LLMs have become gigantic, it has become very hard to get allocated cloud compute to be training or deploying AI models. So it makes so much sense to me that you can have now a local box that is just for you.

00:19:59
You can run whatever experiment you want, you can learn on it. You can also do really heavy lifting, especially when you’re talking about something like, so if you have 96 gigs on each one of these Blackwell chips, you can fit four into a Pro Max. You’re talking about almost 400 gigs. I’m putting you guys on the spot here, but do you happen to know what that would correspond to in terms of model weights in an LLM?

Logan Lawler: 00:20:20
So I’ll give you general as long as you don’t hold me to it and no one listening comes after. As a general rule of thumb, every billion parameters in a model requires two gigabytes. So that in essence say an eight billion parameter, approximately. There’s some things like quantization, what precision you’re running at, FP64, FP4. It all varies.

00:20:42
I think you hit on a huge point is that the model sizes, like say Llama 3-405, before not really possible to run on a workstation. It wasn’t that it didn’t want to or didn’t have the desire, just the technology wasn’t there. I think you’ve hit on a great point is the technology between Dell Pro Max as well as NVIDIA Blackwell GPUs is enabling people to do things. That’s kind of a takeaway from GDC. You’re able to do things you were never able to do before, which I think is super cool.

Jon Krohn: 00:21:12
That is really cool. It’s interesting. It is kind of a full circle. I mean, I’m even thinking back to back when I was doing my PhD, which is before really the most recent AI era. My PhD finished in 2012, which is when there was a big explosion of interest in deep learning as a result of AlexNet, the machine vision model released out of Geoff Hinton’s lab at the University of Toronto. Then all of a sudden everybody in academia and then in industry was taking notice of deep learning.

00:21:39
Actually it’s interesting to have someone from NVIDIA on this call because it’s the insight from Jensen Wong or whoever at NVIDIA at that time to say, “Whoa, we’re building graphics processing units for rendering video game graphics or allowing editors to do video editing, that kind of thing. We’re going to invest a huge amount of money and time and hiring in specializing in this deep learning revolution that seems to be coming.”

00:22:11
So the story that I was going to tell, I’m just going to really quickly wrap that up, but back then, in the pre-AI era, I remember I was working at the University of Oxford. We’d have servers that lots of us would be competing for. It’s cool now to think that in that same lab, people could be buying. I mean, I had a very generous… As part of my PhD, I could spend something like 20,000 pounds, so about $25,000 on hardware. So I could be buying one or more of these workstations and be locally in full control of any LLM stuff that I want to do. So it’s a very cool world that we’re in.

00:22:50
Then I want to get back to the NVIDIA story from around the time and this visionary nature of what NVIDIA’s done and reflected in their share price is this idea that, okay, deep learning’s going to be gigantic or let’s assume that deep learning’s going to be gigantic. So let’s build a software ecosystem. Going back to your point earlier, Sama, that supports that. So yeah, so tell us about things like CUDA, TensorRT, maybe a bit of the history and why those are so important in this GPU ecosystem and in this AI era.

Sama Bali: 00:23:19
Yep. I’m actually going to start first with NVIDIA AI Enterprise, just completing the story of how we are doing things, especially with Dell Pro Max AI PCs. So think of NVIDIA AI Enterprise as our version of end-to-end software development platform, which is helping you not just accelerate your data science pipelines, but also really helping you build next-gen. It can be generative AI applications, it could be computer vision applications, it can be speech AI applications, and it has a lot of components. We’ve got NIM microservices.

00:23:52
This is how we are delivering all kinds of AI models as containerized microservices. So literally think of any other AI model in the world. We work with open source partners, proprietary partners. We have our own NVIDIA AI models as well. We are taking each of these AI models, putting them into a container and then adding our, I wouldn’t say secret sauce because everybody knows about TensorRT LLM and all kinds of services, which are really helping you get the best inference possible on NVIDIA GPUs.

00:24:28
We are offering them as microservices. The reason being, and you’ll soon start seeing this from an NVIDIA perspective that we are providing almost all of our AI software as microservices is because things are changing quickly. I’m a developer today who built an application with Llama 3, and guess what? In two months Llama 3.1 comes and then another two months, 3.2 comes up.

00:24:50
So we want to make it really, really easy for people to just swap in the model as quickly as possible without really disrupting that entire pipeline. So that’s NIM microservices. We’ve gotten all kinds of models from if you want to build a digital human to actually building speech related applications to now. We also have NIM microservices for our reasoning AI models as well. So that’s the first component of NVIDIA AI Enterprise.

Jon Krohn: 00:25:16
Really quickly before. So it’s going to be obvious for sure to you, to both of you, as well as to many of our listeners, exactly what a microservice is, but could you define that for our listeners that don’t know, just so that they understand what it is and why it’s important, why it’s helpful?

Sama Bali: 00:25:34
I actually don’t have a definition of microservice.

Logan Lawler: 00:25:36
I’m going to give you not a textbook definition, but I’m going to give you a practical definition.

Jon Krohn: 00:25:42
Cool.

Logan Lawler: 00:25:42
Is let’s say you’re a data scientist and you have created, let’s just pretend, a chatbot with Llama 3. You create that without a microservice, without an NVIDIA NIM, like Sama said, every time that model updates, if there’s security, all this stuff, you’re doing a ton of, I hate to say it, but background tedious work to get that to a point where you can deploy it.

00:26:09
Where when things change, for example, if you don’t, that’s the whole point of a microservice with NIM is you basically can load that to literally one line of code and the LLM part of it is really done for you. It is containerized, it’s packaged, it’s ready to go. So a data scientist can focus on, well, how am I going to customize it or building whatever application wrapper around it. Versus like, I need to update the code here to get this to connect.

 00:26:33
That’s really the point of a NIM is how quickly can I leverage the power of an LLM vision model, whatever, with one line of code. That’s the power of a NIM. It runs on a workstation too. It runs on Dell Pro Max servers. It runs pretty much everywhere.

Sama Bali: 00:26:48
Yeah, that was going to be my point. That the key point being with these NIM microservices, you don’t have to make sure that the AI model is tuned to the GPU. We’ve done all of that work for you. So as soon as you’re downloading this locally on your Dell Pro Max PC, it already understands the kind of GPU it’s running on. The only thing you have to make sure is the model you’re downloading fits onto your GPU memory size now, but with 96 of memory, you’ve got the entire world for you here.

Jon Krohn: 00:27:17
Nice. So as you’ve been speaking, I’ve tried to look up quickly online what NIM stands for. It doesn’t seem to stand for anything that I can find easily.

Sama Bali: 00:27:26
Okay. I’m going to let the secrets out. It actually stands for NVIDIA Inference Microservice, but then we also use NIM Microservice. So it’s like chai tea kind of a thing. They mean the same thing.

Logan Lawler: 00:27:42
Potato, potato, yep.

Jon Krohn: 00:27:43
Potato, potato.

Logan Lawler: 00:27:44
Potato, potato.

Jon Krohn: 00:27:44
A potato brand potato.

Logan Lawler: 00:27:46
Exactly. Cheese queso. That’s what I would say. I go to a restaurant, I’ll say I want cheese queso, and then my wife always gives me a hard time. Yeah, cheese queso.

Jon Krohn: 00:27:53
Nice. Yeah, now I understand perfectly. Thank you for giving us that insight. It is interesting. It isn’t something that’s very public, so people really are getting the inside scoop on NIM. Yeah, it’s just spelled N-I-M for our listeners who are wondering what word we’re saying. It’s exactly like it sounds. N-I-M, in all caps. I’ll have a link to that in the show notes, of course. Anyway, so I interrupted you. Oh, go ahead, Sama.

Sama Bali: 00:28:15
Oh, I was just on the same topic of NIM microservices, I was going to say, we’ve got a website called build.nvidia.com. That’s where we host all of these NIM microservices. It’s a good website to not just go try out these different kinds of AI models. You have the ability to prototype on the website itself. There are no charges for it at all. You can see models, again, by all kinds of partners that you work with, including NVIDIA models as well.

00:28:43
They’re segregated by the industry you work with or the use case you’re trying to build. So it’s easy to maneuver around, find the exact model you want to work with. Then once you want to download this, we’ve made it easier. So if you really sign up for our NVIDIA developer program, we actually let you download these models and then continue to do your testing, experimentation, free of cost. There are no charges at all. So you can continue as a developer. I would want to go try out different kinds of models, see what’s working with my application. So we like to do that as well.

Jon Krohn: 00:29:15
Ready to take your knowledge in machine learning and AI to the next level? Join SuperDataScience and access an ever-growing library of over 40 courses and 200 hours of content. From beginners to advanced professionals, SuperDataScience has tailored programs just for you, including content on large language models, gradient boosting and AI. With 17 unique career paths to help you navigate the courses, you will stay focused on your goal. Whether you aim to become a machine learning engineer, a generative AI expert, or simply add data skills to your career, SuperDataScience has you covered. Start your 14-day free trial today at www.superdatascience.com.

00:29:55
Fantastic. That was a great rundown. What I was going to say, and I’m glad that you had more to say on NIM microservices because my transition was going to be that the last time I interrupted you, you were about to I think start talking about other aspects of the AI enterprise. So now I’ll let you go on that.

Sama Bali: 00:30:11
So outside of NIM microservices, we’ve got NeMo. Which really helps you build, train, fine tune your own models, but also gets you the ability to add guardrails to your model so that whenever you’re deploying your application, you are making sure that the application gets used exactly the way that you want to do it itself.

00:30:30
We’ve got AI blueprints, think of these as reference AI workflows. So we give you the ability to build different kinds of AI applications. So think of this as a recipe. You’ve got the step-by-step process to actually build an application. There’s a reference architecture, but we also get you the ability to add your own data to it. That’s what gets every company their own edge. You want to add your data, which is your differentiation at this point in time. So you have the ability to build different kinds of applications.

00:31:00
What else do we have? Oh, we’ve got different kinds of frameworks and tools. So we actually do support different kinds of AI frameworks like PyTorch, TensorFlow, we also have our CUDA library. So I think this is a good time to talk about CUDA as well, which really stands for Compute Unified Device Architecture.

Jon Krohn: 00:31:20
I didn’t know that. I didn’t know that. I’ve been using that word for a decade now. Thank you.

Sama Bali: 00:31:25
So this really has been playing a crucial role in AI development by enabling efficient parallel computing on NVIDIA GPUs. So the idea was its entire architecture really helps you train different kinds of models significantly faster, which means that you can in some scenarios actually reduce your training times from weeks to days. It is also helping you get better and better inference.

00:31:56
So you see higher inference performance on NVIDIA GPUs because of this architecture of parallel processing if you’re comparing it to just CPU-only platforms. We now have, and I’ll have to look up the right number of how many CUDA libraries we have, but we’ve got tons and tons of these CUDA libraries. These are GPU accelerated libraries.

00:32:19
So a good example I’ll give you is of RAPIDS cuDF. So the idea, and Logan touched on this earlier as well, is the way RAPIDS cuDF works is that it tends to mimic the APIs of a lot of data frames like Pandas, Polars. So if you are in that process of pre-processing your data in your data science workflow, it can actually accelerate that entire process by a hundred X on our 6,000 GPUs without any kind of code change.

00:32:56
That’s the beauty of it. That as a data scientist, all I’m doing is adding that one API line of code and then it actually accelerates entire process by a hundred X. So that’s massive time saving from a data scientist perspective.

00:33:12
At GTC, we announced cuML, which is again one of our CUDA libraries as well. This is helping you accelerate your machine learning tasks as well. So if you’re using scikit-learn, you have the ability to go up to 50 X acceleration for your ML tasks as well. So each one of these libraries, and as I said, we’ve got tons of these right now, but depending on the data science tasks that you’re doing, these are all designed to then offload that work to the GPU so that you can see that massive acceleration.

Jon Krohn: 00:33:42
Nice. cuML is a new one to me as well. I inferred correctly that the Cu isn’t like the letter Q, it’s like the beginning of CUDA, and so it’s C-U-M-L. Yeah, I’ll have a link to that in the show notes. It looks really cool. GP accelerated machine learning algorithms designed for our listeners designed for data science tasks.

00:34:03
Thank you for the tour, Sama, of all of the amazing things that NVIDIA is doing on the software front for people who are training and deploying ML models. Logan, can you fill us in on how that relates to the Pro Max systems that you are so involved with?

Logan Lawler: 00:34:18
Yeah, I mean, absolutely. So as we talked about before, all the new NVIDIA Blackwell GPU architecture, the 6,000 on the way down really is designed for Dell Pro Max, purpose-built for Dell Pro Max. It’s a little bit beyond Dell Pro Max, but let me give you a perfect example. Like Sama was talking about AI enterprise, which is really at the heart of any data science workflow.

00:34:43
Well, at Dell we sell PCs, but we also sell servers. Where it really fits and ties in, it’s two parts. One is that if you’re using, for example, a Dell Pro Max T2, you can do that work, AI enterprise, clean your data sets, refine, do some fine-tuning, experimentation, all of that, leveraging QDF. Everything through there. Let’s say you want to go then deploy that.

00:35:05
That is where it becomes very seamless using AI enterprise to take it from a Dell Pro Max to the in-house server for deployment or taking it up for bigger experimentation. That’s really the layer that connects everything that we do from the desk side all the way to the data center, which makes it very seamless.

00:35:26
Then I’d be remiss not to talk about, we talked about, we’ll probably talk about it more, is the Dell Max systems that are really purpose built for developers and data scientists being the GB10 and the GB300. Those where if you were to buy a Dell Pro Max T2, there is a cost of AI enterprise. If you look at those two systems, all of those are designed really with all the NVIDIA stuff preloaded ready to go. So it’s out of the box, you plug it in and you’re off to the races.

00:35:56
That’s very different than the other Dell Pro Max systems where I would say maybe not too technical, but let’s say someone here is a data scientists for media and entertainment company. They’re out there, they exist. If you’re doing anything outside of data science, you’re going to want to be in a traditional Dell Pro Max system.

00:36:16
If you’re doing only data science, that’s where you’re looking Dell Pro Max GB10 or GB300 because for example, Creative Cloud doesn’t really work with Linux. It’s just not designed for it. So you have to really make that distinction. That software package is kind of the connecting glue from desk side all the way to deployment, whether you’re doing it on a server, cloud, et cetera.

Jon Krohn: 00:36:36
Okay, so let’s get… Oh, sorry, go ahead, Sama.

Sama Bali: 00:36:38
I was going to do your part, Jon, and I’m going to ask Logan to actually describe how GB10 looks like, like Dell Pro Max GB10.

Jon Krohn: 00:36:45
Cool. Yeah, let’s do that. I have got one thing that I want to get before we start talking about these specific PCs. You gave an example there of something that it was a creative suite. What was it that wouldn’t run on Linux?

Logan Lawler: 00:37:02
Yeah, so perfect example. So Adobe Creative Cloud, which is think Premiere Pro photo editing, video editing. All of that is, that’s something that’s unique is that all of our precision workstations previously or workstations in general could do data science or they could do any other traditional media, entertainment, workflow, video editing, et cetera.

00:37:24
There is a line in the sand which I think would probably make most developers on this call and data scientists call very happy is this is a purpose-built system. The people have to kind of think like, “Hey, if I’m only doing data science, we really need to lean towards the GB10 or the GB300.” If I’m doing anything on top of data science, then I really need to be, because you have to kind of look… That system only comes with Linux, which like NVIDIA, GDX, Linux.

Jon Krohn: 00:37:48
I see. GB10s, GB300s, which are models of Dell Pro Max, which we’ll talk about in a second. Those are Linux-based systems. That is really interesting to hear because, so the question that I had in my head that was starting to come up was I have personally been programming or doing data science on Unix-based systems for a long time now.

00:38:09
So that is an interesting… What I was going to ask and actually, so here is still a question that I think is hopefully really interesting for a lot of listeners and is definitely interesting for me is why should I consider switching from a Unix-based system to perhaps a Windows-based system as a data scientist?

Logan Lawler: 00:38:31
Well, I mean a couple of things, is that you don’t necessarily have to with the GB10 and the GB300 because that is Linux-based. That’s why. At the end of the day, we know, I mean, I’m not a data scientist, never claimed to be one, but you all love Linux and that’s great. It works well, accelerates well. I’m a Windows guy. I’ve always been a Windows guy. I mean, there is some optimizations, not that we can really talk about, but with the WSL2, it makes that seamless transition a little bit better.

00:38:57
I personally like Windows. Not trying to change your mind, but if you’re doing anything outside of that, it does sometimes make it easier to run on Windows depending on the applications that you’re actually using and just from a compatibility standpoint. Yeah, if you’re a data scientist, you like Linux. That’s the whole reason for GB10 and GB300 is you like it it works. You’re used to it. There you go.

Sama Bali: 00:39:20
I would say you don’t even have to choose, especially with GB10. I’m going to steal your thunder here, Logan. It literally fits in the palm of your hand, Jon. It’s this tiny small box. Logan actually has maybe [inaudible].

Logan Lawler: 00:39:32
I had one, I brought one back from GTC, but this is a representative example, probably actually bigger, box of Kleenexes. Probably chop off that.

Sama Bali: 00:39:42
Exactly.

Logan Lawler: 00:39:43
I mean legit. Palm of the hand.

Jon Krohn: 00:39:47
That’s actually useful given that most of our listeners are only listeners and not viewers of YouTube. It’s actually maybe even more useful that you just brought out a Kleenex box.

Logan Lawler: 00:39:55
That’s all I had.

Jon Krohn: 00:39:57
That gives our audio only listeners an idea that he took a Kleenex box and cut off about a third of it.

Logan Lawler: 00:40:03
Yeah.

Sama Bali: 00:40:04
Going back to my point, you can actually be using a Dell Pro Max AI PC with Windows running for all of the productivity apps and have this GB10 attached to it. You can let it be, if you’re training a small model, fine tune it, it can be on the side.

Logan Lawler: 00:40:18
You can actually daisy chain two of them together. So you can actually have a Dell Pro Max, a laptop if you’re a mobility person and have two GB10s network together. Is it NVIDIA X Connect? Or whatever the connection is.

Sama Bali: 00:40:31
ConnectX, yep.

Logan Lawler: 00:40:33
ConnectX. Yeah. So you can actually have two, and it’s smaller than… Pretty much even that setup would be smaller than pretty much every desktop tower that we sell.

Sama Bali: 00:40:44
That gets you the ability of not… You don’t have to choose, you can continue doing all of your productivity workflows on Windows if that’s what you’re used to. Then if you’re also doing your data science or developer tasks, you can easily do that on Linux by having best of the both worlds together.

Jon Krohn: 00:40:59
So this is similar to the situation that I was describing earlier where I was talking about, hey, I hand built these servers with NVIDIA GTX 1080ti in them years ago, and four of us on the data science team would log in through a terminal. So that’s a similar idea here where you can be using whatever operating system you want on say the laptop you’re typing on.

00:41:18
Then when you want to do something with an LLM, you open up a terminal window, some kind of window for accessing that machine and you run from there. Okay, that’s really cool. So now we understand that a GB10 you can blow your nose with it and it’s very compact. Tell us about the GB300, the other Linux based.

Logan Lawler: 00:41:40
So the GB300, you are not blowing your nose with because it is, I don’t really have a representative example, but it is a traditional sized fixed tower. What I think is so exciting about this is, one, it’s running on the Grace Blackwell ultra super chip. So when we say GB, Grace Blackwell.

00:42:01
So this is a kind of an integrated system on a chip design that has 784 gigs of unified memory, 288 specifically for GPU, 496, hopefully my math is correct of CPU memory. TOPS wise, I know that that’s a term maybe audiences heard of or not. It’s basically tera operations per second is at FP4, that is 20,000 TOPS. Let me just give you a set context of that. Is that within the RTX cards, the Blackwell cards, just the singular 6000, 96 gigs, that’s about 4,000 tops approximately.

00:42:40
So this is a very, very server level, powerful system that is designed for, I mean you could really put this thing in a data center and it would act like a server. It is at the desk side and the power and just the whole thing and the whole design, I think it’s not going to be for everyone at the end of the day. For those that are heavy data scientists working in an enterprise, this is going to be a system of choice for you. Just, I mean honestly with the horsepower that it’s built, it’s kind of insane to be completely honest.

Sama Bali: 00:43:15
I actually see teams using this where multiple people can access at the same time. Jon, the way you were talking about how you had a small team of AI developers and data scientists, they could be using it at the same time, plugging in, getting their work done. It’s again, on the desk side. It’s on prem, you get to keep your data private right there with you.

Logan Lawler: 00:43:39
Exactly. I want to add onto that because that’s a good point. Is that the multi instances of that, it’s very important is that at the end of the day, just being very transparent from the knowledge that I’ve gained, is that when you look at companies that have started down an AI path or journey, it’s usually the bigger companies because there’s cost associated, that takes time, talent. What I think, and then just generally, I’m not a server guy, I’ve never been a server guy. It is not that they’re bad, I just don’t know much about them.

00:44:11
That’s a skillset that I don’t have. Everyone has a client, desktop skillset. I really think what the GB300, like to Sama’s point will bring is if you are maybe in a smaller company, you’re more of a mid-market and you don’t have servers and you don’t want to mess with it, this gives you the ability to really bring AI to your company. Whether it’s RAG model, fine tune something, build up some agentic apps, whatever you want to do. You’re not having to go out and get racks and cooling and all the other things that come along with servers.

Jon Krohn: 00:44:43
This episode is sponsored by Adverity, an integrated data platform for connecting, managing, and using your data at scale. Imagine being able to ask your data a question, just like you would a colleague, and getting an answer instantly. No more digging through dashboards, waiting on reports, or dealing with complex BI tools. Just the insights you need – right when you need them. With Adverity’s AI-powered Data Conversations, marketers will finally talk to their data in plain English. Get instant answers, make smarter decisions, collaborate more easily—and cut reporting time in half. What questions will you ask? To learn more, check out the show notes or visit www.adverity.com.

00:45:27
Amazing. That is actually exactly the next question I was going to ask was around cost and when it would be appropriate to use servers or this kind of system. So you just nailed it there. I don’t know if anyone else has anything to add on that. This builds on the point I was making earlier about this transition from, there was a point when I think back to my PhD ending in 2012, there were a fair few number of things that I could at least test locally.

00:45:55
Maybe I’d be like, okay, now I’m going to scale this up over a larger data set and so I’ll use a super compute cluster or something. In the world that we’re in now, it’s a completely different world where I can’t just have a gigantic Llama model on my laptop and do anything with it. It’s impossible.

00:46:11
So it is an interesting… It’s a completely different kind of world that we’re in now. So we went through this transition from being able to do a lot of things as a data scientist locally on a laptop, then to getting used to doing a lot of things in the cloud.

00:46:28
Now folks like you two literally are bringing new kinds of solutions, a new kind of paradigm where you can have either a Kleenex box or a desktop tower that can superpower, supercharge your ability to be using all the cutting edge AI models yourself without having to wait in line for anyone else. So I guess I just summarized a lot of points there, but I don’t know if you guys have anything to add around who this is a great solution for in terms of a company or an individual particularly maybe with respect to cost or efficiency.

Logan Lawler: 00:47:00
I mean, I’ll take a stab at it. So GB10, Kleenex box, we’re just going to call it the Kleenex box. I mean that could be scaled down to a student or it could be scaled up and put inside a Dell. I personally am going to get one and use it.

00:47:15
The GB300 is really, regardless of enterprise size, it’s people that are doing very, very heavy workloads. I mean, it is a server

[inaudible 00:47:27].
Now pricing is TBD, so I’ll probably get in trouble. Let’s just say that NVIDIA and their DGX Spark product, I think, is it 3,000 to 4,000, Sama? Is that right?

Sama Bali: 00:47:39
Yep.

Logan Lawler: 00:47:40
Okay. I would assume that Dell’s probably in that range for the Kleenex box as well. Pricing hasn’t been released for GB300, but considering you’re going from a thousand AI TOPS all the way up to 20,000, it will be more expensive, but it will not be the cost of a server.

Jon Krohn: 00:47:56
Cool. That’s a great…

Logan Lawler: 00:47:58
That’s a way to keep me out of trouble. Way to keep me out of trouble.

Jon Krohn: 00:48:02
Really nice. Yeah, I was wondering if we get into specific pricing. I think that was directional enough that it’s helpful to me and our listeners. All right, so one last big technical question for both of you. You guys are sitting right at the forefront of AI.

00:48:16
You’re going to conferences like GTC, which also I should clarify again just for listeners when, anytime on this show that anybody said last week or today, they’re talking about at the time of recording, which is about a month before this episode is published. So just so you’re aware that you’re not on a time warp. GTC was whatever five weeks ago, not last week when you’re listening to this.

00:48:39
So you’re on the ground that both of you have, where do you see… I realize it’s very hard to look into a crystal ball with how fast AI moves. If you could try to make some predictions about what being a data scientist or an AI developer or maybe even just what life will be like in the coming years, the coming decades. I would love to have your thoughts. Big question. I don’t know who wants to go first. You can just rush into that.

Logan Lawler: 00:49:13
I mean, I’ll take a bit of a stab at it. I mean, I have wild theories about what the world might be like with the advent of AI. I don’t think it’s going to be robots taking over. I don’t believe it’s that. I believe is that if you think about it, any sort of technology or software, there’s some sort of seminal moment where think about first time you had a cell phone in the palm of your hand.

00:49:42
Yes, there is AI in different applications and stuff like that. I think what you’re going to start seeing is two big shifts. Is that one, you’re going to see AI get into the hands of a lot more people. What I mean by that is not, and I’m going to be… I am not a developer. I mean Sama will attest.

00:50:02
When I was hired for the shop about a year ago, I mean I was not great. I was able to go out, learn, educate myself, pull down different SDKs from GitHub, other things, to go out and train my own [inaudible] for an animation studio. I was able to do X, Y, Z. I think you’re going to see that become a lot more accessible and easy and popular as the years go on.

00:50:27
Then I think the other thing that you’re going to start seeing is AI make its way into our daily life. For example, complete crazy, but my mom is old school, they’re like 70, but she has a recipe book in these little recipe cards thing. I was like, “Hey, Mom, you know that pumpkin pie recipe that you have? I’d really like to get it.” She’s like, “Oh, let me find the card.” I mean, there’s thousands of cards, dude. That’s not alphabetized or whatever.

00:50:56
I was like, wow, what if my mom had a RAG model of all of her recipes where all she had to do was really type that in and just say pumpkin pie and it would just deliver and be able to tell you that. Could we go set that up? I could go set that up for her, but that would be the work. I think what you’re going to start seeing is some of these technologies and things like that make its way into the mainstream where it’s going to simplify our lives. You know what I mean? I think that that’s what you’re going to start seeing over time.

Sama Bali: 00:51:24
I’m going to repeat what Jensen kind of painted that picture in his keynote as well. That we’ve gone from really the years of generative AI to now being in the world of agentic AI. You’ve got an agent, an AI powered agent for everything. So if you are, let’s say in a factory setting, you’ve got an AI agent, which is managing your incoming raw material and how much that is coming. Let’s say you’ve got less raw material.

00:51:50
So this AI agent is telling that AI agent, which is managing the floor, guess what? We’ve got less raw material. So your end product is going to be less. Then this AI agent itself is telling the transportation one that we’ve got less. You don’t need that many trucks today. We definitely are entering that world with a lot of these reasoning AI models coming into being as well of AI agents where you can build these systems which have the ability to learn, perceive, but then also act.

00:52:20
I think what the future is, is all about physical AI. You have a lot of these autonomous systems now which are able to again, learn, perceive, but then accordingly act. This is in our physical world itself. So if you think of autonomous vehicles, I am in the Bay Area, I get to see a lot of these driverless cars all the time. Then every week I see them getting better and better at it because they’re learning.

00:52:47
They’re perceiving different kinds of conditions of the road, if they’re seeing somebody walking on the streets as well. So I definitely bet AI can get integrated a lot with our physical world. As a personal opinion, I hope there are a lot of guardrails and regulations just to make it safer for everybody to use it then.

Jon Krohn: 00:53:09
That was well done, and I realize you’re using the CEO of NVIDIA, which is this hugely important AI company globally. So I really appreciate you bringing those insights. It makes it so clear to see where we’re going. I think you two worked perfectly, not only on that question, but on this whole episode. I’ve loved this. It was so much fun and I hope that I can get both of you on an episode again sometime soon.
Logan Lawler:

00:53:39
We’ll make it happen.

Jon Krohn: 00:53:41
Yeah, to dig into another topic.

Sama Bali: 00:53:41
High five, Logan.

Jon Krohn: 00:53:41
Yeah, nice.

Logan Lawler: 00:53:41
We nailed it.

Sama Bali: 00:53:41
Yep.

Jon Krohn: 00:53:44
Before I let you guys go, I always ask my guests for a book recommendation. Have you got one for me? Maybe, Sama, go first. I think I saw you hold a book up earlier.

Sama Bali: 00:53:52
Yes, this is definitely one of my favorite book. It’s called The Forest of Enchantments. It’s by this Indian author called Chitra Banerjee. I highly recommend this. It’s based on Ramayana, the mythological story around it. It’s great, especially for any, I want to say woman who’s trying to come up, create their own career in any field. It gets you a lot of self-confidence in you as well.

00:54:19
I do love Gary Janetti, if he’s listening. I am a really, really big fan. His books are amazing. They make me laugh. They make me cry at the same time. So I am a very big Gary Janetti fan. I love all his books. So those would be my recommendations.

Jon Krohn: 00:54:34
I’m sure famed author Gary Janetti is a listener to the SuperDataScience podcast. No question. I’m sure he’s really happy to hear it. Thanks for those great recommendations, Sama. Logan, what have you got for us?

Logan Lawler: 00:54:47
I thought about this and I’m not a huge reader. I’m more of a talker.

Jon Krohn: 00:54:54
You’re a good talker.

Logan Lawler: 00:54:54
You probably can’t see this, but I’ve got my Get Started manual for my Dell GV, my XB webcam. It’s riveting reading. No, I’m just kidding. Books wise, it’s been a while. I’ll admit. One book that I do do not have in front of me like Sama, but I really did like, and as cliche as it might be, was the Art of War by Sun Tzu, like in the context of business. I’ve read it over the course of a couple of times.

00:55:28
It was just, I like it because it definitely relates back to business and things about the importance of strategy and thinking about what you’re doing before you go out and tackle things and talking about, hey, what is a tactical way I want to do things. Or thinking about, hey, when I’m under fire, how do I handle that? Not that promoting war or anything like that, but just in the context of business it’s very interesting. I’ve definitely used a few nuggets from that book over the course of my career, for sure.

Jon Krohn: 00:55:58
Awesome, great recommendation. I’ve read some excerpts from it, and it does seem…

Logan Lawler: 00:56:03
It’s not a very long book.

Jon Krohn: 00:56:06
I’ve heard that as well.

Logan Lawler: 00:56:07
It’s about the Dell Pro Max webcam. It’s pretty close, pretty thin like a tissue. If we’re keeping on the tissue box analogy, it’s about a tissue.

Jon Krohn: 00:56:16
How many arts of war can you fit into a single GB10?

Logan Lawler: 00:56:22
Probably three, paperback.

Jon Krohn: 00:56:27
Yeah. Nice. Awesome. Thanks so much, both of you. If people want to hear more insights from you or just have a laugh perhaps, because you’ve both been really fun and funny on the show today, how can people follow you? Sama, do you want to go first?

Sama Bali: 00:56:41
I think the best way is LinkedIn. I’m probably the only Sama Bali you will find there, so that’s the best way. Send me a message. I’m happy to connect.

Jon Krohn: 00:56:49
Nice. Yeah, we’ll have a link to your LinkedIn in the show notes and maybe Logan’s too.

Logan Lawler: 00:56:55
Yeah, I mean mine as well. LinkedIn’s the best. That’s where I’m most active. It’s just Logan Lawler. I mean, pretty straightforward. I think I’m the only one. Actually, no, there is a couple of other ones, which is shocking. I’m the one, yeah, that looks like this.

Jon Krohn: 00:57:11
At Dell.

Logan Lawler: 00:57:11
Yeah. The one at Dell, that says Dell Pro Max in their profile.

Jon Krohn: 00:57:14
Yeah, for our audio only listeners, find the LinkedIn profile that sounds like this.

Logan Lawler: 00:57:20
Yeah, exactly. Exactly.

Jon Krohn: 00:57:23
Perfect.

Logan Lawler: 00:57:23
Hey, Jon, one other thing I want to talk about before we close out and we really appreciate you having us on the show is that for everyone that’s listening is I’m putting together a program. Easiest way to describe it is we’re looking for data scientists that have heavy industry knowledge experience to help us do two things.

00:57:42
One, to help us test products, the GB10, the GB300, as well as in our product design process where at Dell we work directly with the ITDM’s, but we’re always looking for the end users that are actually using the workflows, whether it be in data science or M&E or engineering. It doesn’t really matter. Being able to understand the workflows that impact them, what they do on a daily basis and what they need in a Dell Pro Max. So I’m starting a developer advisory council, so we say.

00:58:13
If you’re interested in it, want to hear more, I’m throwing out the offer. Happy to chat. I’m very fast at email. It’s just Logan_Lawler@Dell.com and it’ll also be, Jon I think said, we’ll include it in the show notes. Reach out to me. Would love to hear it, would love to meet you, hear about what you’re doing. If it’s a right fit and it makes sense, then would love to have you a part of the council.

Jon Krohn: 00:58:33
Love that. That’s such a nice call to action for my audience who should be spot on the money for that kind of thing.

Logan Lawler: 00:58:40
Well, that’s why I need them. It’s not me. It’s not me.

Jon Krohn: 00:58:42
It’s perfect. Thanks, Logan.

Logan Lawler: 00:58:44
Of course.

Jon Krohn: 00:58:45
All right, so awesome having both of you on the show. Yeah, really enjoyed it as I’ve already said. Yeah, and hopefully catch you again soon.

Sama Bali: 00:58:54
Thank you for having us.
L

ogan Lawler: 00:58:55
Thanks, man. Appreciate it, Jon. It was a great time.

Jon Krohn: 00:58:57
What a fun, informative time with Sama Bali and Logan Lawler. In today’s episode, they covered NVIDIA’s Blackwell GPUs and how they offer unprecedented memory capacity of up to 96 gigabytes per GPU with the ability to install four GPUs in one workstation, providing nearly 400 gigabytes of GPU memory. Sufficient to run LLMs with about 200 billion parameters.

00:59:27
They also talked about how NVIDIA AI Enterprise software creates a seamless ecosystem between workstations and servers, featuring NIM, NVIDIA Inference Microservices that allow one-line implementation of AI models without requiring manual GPU tuning.

00:59:41
They filled this in on Dell Pro Max PCs and how they are designed specifically for heavy computational workloads that require GPU acceleration and a UNIX-based operating system. They talked about how the GB10 workstation is compact enough to fit in a palm, yet powerful enough for significant AI workloads. While the GB300 delivers server-class performance with 20,000 AI TOPS and nearly 800 gigs of unified memory.

01:00:05
Finally, we talked about how the democratization of AI is accelerating as technologies like these make powerful AI capabilities accessible to smaller organizations and individuals without requiring enterprise-level server infrastructure.

01:00:19
As always, you can get all the show notes including the transcript for this episode, the video recording, any materials mentioned on the show, the URLs for Sama and Logan’s social media profiles, as well as my own at SuperDataScience.com/883.

01:00:33
If you’d like to engage with me in person as opposed to just through social media or this podcast, I’d love to meet you in real life at the Open Data Science Conference, ODSC East, which is running from May 13th to 15th in Boston. I will be hosting the keynote sessions and along with my longtime friend and colleague, the extraordinary Ed Donner. I’ll be delivering a four-hour hands-on training in Python to demonstrate how you can design, train, and deploy cutting-edge multi-agent AI systems for real-life applications.

01:01:03
Thanks of course to everyone on the SuperDataScience Podcast team and a warm welcome to Nathan Daly who just joined us as our head of partnerships. In addition, we’ve got Natalie Zijajski on partnerships as well, our podcast manager, Sonja Brajovic, our media editor, Mario Pombo, our researcher, Serge Masis, our writer, Dr. Zara Karschay, and our founder Kirill Eremenko. Thanks to all of them for producing another excellent episode for us today.

01:01:29
For enabling that super team to create this free podcast for you, we are deeply grateful to our sponsors. You can support this show by clicking on our sponsors’ links, which are in the show notes. If you yourself would like to sponsor an episode, you can get the details on how at Jonkrohn.com/podcast.

01:01:46
Otherwise, share the episode with folks who might like to listen to it as well, or view it as well. Review the show on your favorite podcasting app or YouTube. Subscribe if you’re not a subscriber, edit the videos into shorts if you want to. Most importantly, just keep on tuning in.

01:02:02
I’m so grateful to have you listening, and I hope I can continue to make episodes you’ll love for years and years to come. Till next time, keep on rocking it out there and I’m looking forward to enjoying another round of the SuperDataScience Podcast with you very soon.