Jon Krohn: 00:00:00
This is episode number 879 with Dr. Greg Michaelson, co-founder of Zerve. Today’s episode is brought to you by Trainium2, the latest AI chip from AWS and by the Dell AI Factory with NVIDIA.
00:00:20
Welcome to the SuperDataScience Podcast, the most listened to podcast in the data science industry. Each week we bring you fun and inspiring people and ideas, exploring the cutting edge of machine learning, AI, and related technologies that are transforming our world for the better. I’m your host, Jon Krohn. Thanks for joining me today. And now let’s make the complex simple.
00:00:41
Welcome back to the SuperDataScience Podcast today, the highly technical, but also highly hilarious Dr. Greg Michaelson returns to this show for the first time. Greg is a co-founder of Zerve, a super cool platform for developing and delivering AI products that launched to the public on this very podcast a little over a year ago.
00:01:14
He previously spent seven years as DataRobot’s chief customer officer and four years as senior director of analytics and research for Travelers Insurance. He was a Baptist pastor while he obtained his PhD in applied statistics from the University of Alabama. Today’s episode is on the technical side and so will appeal most to hands-on practitioners like data scientists, AI or ML engineers and software developers. But Greg is such an engaging communicator that Baptist background coming in handy that anyone interested in how the practice of data science is rapidly being revolutionized may enjoy today’s episode. In it, Greg details how Zerve’s collaborative graph-based coding environment has matured over the past year, including the Revolutionary Fleet feature that allows massive parallelization of code execution without additional cost.
00:02:00
He talks about how AI assistants are changing the coding experience by helping build, edit and connect your data science projects. Why the rise of LLMs might spell trouble for many SaaS businesses as building in-house solutions becomes increasingly viable. And the innovative ways companies are using RAG, retrieval augmented generation to create more powerful AI applications. All right, you ready for this entertaining and educational episode? Let’s go.
00:02:30
Greg, welcome back to the SuperDataScience Podcast. It’s great to have you back. Where are you calling in from this time?
Greg Michaelson: 00:02:36
I’m home. I’m in Elko, Nevada.
Jon Krohn: 00:02:39
Very nice.
Greg Michaelson: 00:02:39
Which is a small town. We’re about three hours west of Salt Lake City.
Jon Krohn: 00:02:43
Cool. So that’s like the nearest city. You’re nowhere near Las Vegas for example?
Greg Michaelson: 00:02:49
Vegas is like eight hours south.
Jon Krohn: 00:02:51
Oh, boy. Yeah.
Greg Michaelson: 00:02:52
Yeah. And then Reno is probably four hours to the west. So we’re about midway between Reno and Salt Lake City. It turns out Elko is the gold mining capital of the US.
Jon Krohn: 00:03:02
Oh, really?
Greg Michaelson: 00:03:03
Not many people have heard of it, but they pull an awful lot of gold out of the ground here.
Jon Krohn: 00:03:07
No kidding. Still today. Still going on.
Greg Michaelson: 00:03:10
That’s cool. Well, we still need it.
Jon Krohn: 00:03:14
We still desperately need it. I guess there are actually real world applications of gold, but I think most of it goes into jewelry, which I guess some people also-
Greg Michaelson: 00:03:23
I don’t know, I think most of it probably goes into electronics.
Jon Krohn: 00:03:26
Oh, yeah, there you go.
Greg Michaelson: 00:03:28
I have no idea. I’m just speaking out of complete ignorance here.
Jon Krohn: 00:03:32
Well, listeners can let us know.
Greg Michaelson: 00:03:34
Dunning Kruger at work here.
Jon Krohn: 00:03:36
Yeah, yeah. We’re not going to do research right now to figure that out, but it’s not data science related enough.
Greg Michaelson: 00:03:43
Yeah, this isn’t Joe Rogan kind of gig.
Jon Krohn: 00:03:47
So what’s been happening since you were on the show a little over a year ago? I hear you have big personal news actually. Nevada appropriate personal news.
Greg Michaelson: 00:03:58
Well, I did get married last week.
Jon Krohn: 00:04:00
Yeah.
Greg Michaelson: 00:04:00
Went down to the courthouse and got hitched for the second time, so-
Jon Krohn: 00:04:05
Yeah. Nice. Congrats. Congrats, Greg.
Greg Michaelson: 00:04:06
Thanks.
Jon Krohn: 00:04:07
That’s big news. So we did have you on, as I said a little over a year ago. That was episode 753. And so in that, our listeners did get an overview of what Zerve was like, but I understand that you would have… You’ve released quite a few new features since then.
Greg Michaelson: 00:04:24
Yeah, it’s been a wild year. We did some real basic stuff like integrating with a GitHub and Bitbucket and stuff like that, but we also have added a feature called the Fleet, which is coming out this week. And that’s a way to massively parallelize code execution using serverless technology. If you wanted to make… Let’s say you wanted to make a call to a large language model, but you wanted to do it say a thousand times, they’re slow, right? Like everybody’s used ChatGPT, you can type it, ask it a question. It might take 20, 30, 40 seconds for it to come back. So if you’re trying to do that a thousand times, doing it in series is a kind of a pain. So you could do multiprocessing or something like that, but then you got to manage the pools and figure… There’s a coding challenge there. Maybe not a challenge for the experts, but it takes more than one line of code.
00:05:21
But Zerve is a block-based, you’ve seen it, the code is arranged as a DAG and so each block when you execute it, spins up serverless compute and executes. Well, it turns out that it’s dead easy to just parallelize by speeding up lots of serverless compute. And the upside is you don’t have to write any code to do it and it doesn’t cost more because it’s the same amount of compute. It just happens all at the same time. So the fleet is pretty awesome. And then our AI assistant that can help you to write code within the app can build its own blocks, can do some of that stuff. It’s sort of modern coding because the large language models really revolutionize the way people code. It’s just not at all the same kind of endeavor as it was. So yeah, some really, really cool stuff.
Jon Krohn: 00:06:17
A hundred percent. And so Zerve in general is about making, creating code faster, easier to understand, easier to collaborate on. So we’ll get into those details like the AI system which accelerates that even more. We’ll talk about the LLMs and that kind of parallelization, the serverless aspects of that as well. But quickly, first we should, for folks who haven’t listened to episode 753, we should fill them in a bit more. So now we know it’s a directed acyclic graph, so maybe we should kind of explain that term a little bit. So for example, it’s a directed acyclic graph, it’s basically what it says, everything’s in there in the term. But to break it down, you have a graph, which means you have nodes and edges connecting those nodes.
00:07:04
It’s directed, which means you have some kind of flow. It’s not just that the points in the graph are connected, meaning like, oh, these are all my friends, we’re connected in a graph. No, it’s directed. There’s some kind of flow of information between all those nodes in the graph. So it’s like outlining a process, like a data science process, a data modeling process, a data engineering process, all that stuff could be nodes and directed edges in your DAG. And then the final term there, the A means that nowhere in your directed graph is there a loop.
Greg Michaelson: 00:07:40
You got it.
00:07:41
Yeah, that’s a good description. Yeah. In Zerve’s graph, we call it a canvas. The code lives in the nodes in your graph and then the connections indicate data and memory flow. So if I execute my first block and it is say a SQL query to a Snowflake database, then the data frame that gets created will pass down that node to the subsequent blocks. And you can fork and go one to two and then you can merge back together and go two to one and so on. The nice thing about the architecture is that you can run as many blocks simultaneously as you want. So you can have multiple people in the same canvas, all running code at the same time, writing code in Python, in R and SQL. It’s just kind of a wild mix and match environment where anybody can do anything. It’s really cool.
Jon Krohn: 00:08:32
That’s one of the cool things about it. It reminds me of the shift and I can’t remember if I said this in 753 or not. So my apologies to listeners who listen to that if I’m recycling this analogy again. But what Zerve reminds me of is the shift from having Microsoft Docs that you emailed around to each other and you completely lose track of what versions you’re on. Somebody’s working on… You have a big legal document and your legal team is making corrections while their legal team is making corrections and you’re reading it and taking notes and then that merger becomes a nightmare. Zerve does the same kind of thing in data science that Google Docs did where you could all of a sudden be collaborating altogether on the same say, legal document. And you can see when somebody’s typing, you can see that they’re leaving a comment. You can do things in real time. You could be potentially on a Zoom call at the same time and talking it over. And so Zerve allows that same kind of interactivity and visibility and parallelization that Google Docs did.
Greg Michaelson: 00:09:33
Yeah. Exactly. So if you’ve ever worked in a Jupyter Notebook then you know that sharing and collaborating on those is a nightmare, right?
Jon Krohn: 00:09:41 And commits-
Greg Michaelson: 00:09:43
Well, every time you open it, the metadata changes. So you end up with tons of merge conflicts and all that stuff, which is a nightmare. And so you end up with files that are like document, final document, final, final, final, document really final this time. Use this one. You know what I mean? So the file naming is hilarious.
Jon Krohn: 00:10:03
Yeah. And so the net effect with a tool like Zerve is that according to your materials… And so you’re going to have to explain this to me. It says that it empowers code first data teams to cut cycle times by up to nine times. So there’s a couple of things that’s interesting about this. So the first thing is code first data teams. It sounded like when you described that DAG that it could be a low code, no-code tool, but in fact we’re talking about empowering code first data teams. So that’s something to clarify first. And then how does Zerve empower code first data teams to cut cycle times on model development up to five? Yeah, basically 10X.
Greg Michaelson: 00:10:38
Well, yeah, code first is definitely not a no-code, low-code tool, although it’s… Large language models are making that a little bit fuzzier. The 2010s was really like the era of the low-code, no-code tool. I was at DataRobot, one of the original low-code, no-code data science tools for, I was there for seven years or so. And I think everybody is pretty well realized that low-code, no-code, the emperor’s got no clothes on that. Anytime you run into a complicated problem, you’re going to exceed the bounds of what you can do in a low-code, no-code tool. And you’re going to have to… And that’s why like they all… Like DataRobot bought a notebook environment, Databricks introduced a notebook environment, they’re all kind of shifting to coding environments because they’re realizing that the only people that actually generate value from data are the experts, the ones who are writing code.
00:11:33
And so we very much started out as a coding environment, a place where you could write code. And we still are. There’s been no pivot. We are a coding environment, but now with large language models, it’s not exactly the same type of thing. You’re not coding the same way that you were, where you’d sit down at a blank screen and just type code. Instead, you’re sort of describing to the large language model what you want to build and then you’re taking that code and maybe you’re tweaking it or maybe you’re interacting with the large language model to make it do what you want it to do. And so it’s still very, very code-based, but it’s just a different experience. It’s really wild how coding has changed over the last… In such a short time over the last two or three years.
Jon Krohn: 00:12:19
For sure. It’s completely transformed. It does allow people who have never code before to suddenly kind of be able to get into that. We had an episode recently with Natalie [inaudible], it’s episode 873. And in that episode she talked about how she never learned how to be a developer, but now she can pick up tools and use something like Claude or ChatGPT and be able to generate code herself. Now it’s interesting because it’s not as turnkey as you might like it to be today, where she described how she kind of, she was like, “Okay, cool, I can just code.”
00:12:55
Then she quickly realized, “Well, you know what? I actually do also need to read an introductory Python book at the same time.”
Greg Michaelson: 00:13:01
Yeah, yeah. There’s little… You couldn’t sit down with your grandma and just do it. But after some time, grandma could pick it up probably. But there’s still value in experts and you still need to know how to read it and stuff like that. Especially, particularly when projects get much more complex. It’s good for getting started but then at some point, we’re not exactly to the place where it’s no expertise required and I don’t think we will be for a while.
Jon Krohn: 00:13:31
And so it’s very easy for me to imagine having a tool like Zerve, you can arrange your directed acyclic graph so that you are taking in some data input or maybe multiple data inputs. You’re doing data pre-processing on each of those streams separately. And because they’re different things like one is weather, another one’s stock prices, another one’s images. You have all these different data inflows that need completely different kinds of data pre-processing. And so it makes sense to see that laid out in Zerve DAG. And then you talked about how you can have multiple streams in that DAG combining into each other or forking away from each other. And so for example, if you have three data inputs, you might want to have them all go into the same models. So you merge them together onto a node and then you have them flow into a neural network or something like that. That’s all easy for me to visualize and probably for our listeners to visualize. So how does the code component manifest in that environment? How do you see it?
Greg Michaelson: 00:14:30
How do you see the code?
Jon Krohn: 00:14:31
Yeah, like how do you like… If you have these nodes, I feel like it’s now a dumb question with the way you reacted, but do you click on a node and then you see some code under the node?
Greg Michaelson: 00:14:42
Sure. So the node itself is a text editor. It’s a code window. We use Monaco. It’s open source. It’s the same code editor that VS Code uses. So each node is a little text window and it turns out when we looked at the way people are using it, they actually go into full screen mode. So for each node you can click into a full screen mode that gives you… Imagine kind of a heads-up display on that particular block. The central area is the actual code. And then on the left side of the screen you have all of the inputs, what data is being fed in from upstream blocks. And then you have all the outputs on the right side of the screen.
00:15:19
So you can actually… One of the cool things about working in Zerve is that if I run my code and I say I do something to a data frame, maybe I convert a variable from character to numeric or something like that, then I can click on it on the left side and have a preview and I can click on it on the right side and have a preview and I can compare what did my code actually do without having to type like printdf.head and all that sort of stuff in order to actually see it. Because if I go back and look at notebooks or projects that I’d worked on in the past, those are everywhere. And this is the whole task around like, okay, how do I take out all my print statements? Because I need to look at my variables. So yeah, it’s neat to be able to preview.
Jon Krohn: 00:16:02
This episode of SuperDataScience is brought to you by AWS Trainium2, the latest-generation AI chip from AWS. AWS Trainium2 instances deliver 20.8 petaflops of compute, while the new Trainium2 UltraServers combine 64 chips to achieve over 83 petaflops in a single node – purpose-built for today’s largest AI models. These instances offer 30-40% better price performance relative to GPU alternatives. That’s why companies across the spectrum – from giants like Anthropic and Databricks to cutting-edge startups like Poolside – are choosing Trainium2 to power their next generation of AI workloads. Learn how AWS Trainium2 can transform your AI workloads through the links in our show notes. All right, now back to our show.
00:16:53
So let’s talk more about the LLM aspect of this now. So you kind of got into it a little bit there. Many companies are struggling with leveraging large language models into their businesses and part of why that is a struggle for them, despite the obvious value that you get as a data scientist or a software developer when you’re working on your own projects outside of a business. And part of what makes that tricky for companies is that they’re worried about their intellectual property. They’re worried about sending off their company software, potentially if they’re a software company, the most valuable IP just sending that off to openAI-
Greg Michaelson: 00:17:33
In a prompt. Yeah. In a prompt.
Jon Krohn: 00:17:34
Yeah, exactly. Yeah.
Greg Michaelson: 00:17:36
And they should be.
Jon Krohn: 00:17:37
Yeah. So how do you resolve that with now your integration of LLMs into the product? And actually it would be helpful to know in the same way that you just… Here’s a term, I just realized is as node and code rhyme so nicely with each other, you talk about no code environments. You’ve created a node code environment.
Greg Michaelson: 00:17:59
I appreciate that so much actually. It’s a code node.
Jon Krohn: 00:18:03
Yeah. And so then how do you integrate LLMs into your code nodes? How do those manifest? How do you experience them? And then how can companies feel comfortable using those?
Greg Michaelson: 00:18:16
Yeah, so there’s a lot to say there. There are really three options at the moment inside of Zerve for interacting with large language models. Well, four really. The first is OpenAI. So at the end of the day, OpenAI’s models are the best. They’re better than all the open source ones in terms… At least in my experience. I’m not sure if I believe all the benchmarks that are out there about evaluating performance of LLMs and which ones give better answers and all that sort of thing. But anyway, so we have OpenAI and if you interact with ChatGPT, then you are sending stuff to OpenAI and there’s kind of no way around that.
00:18:53
So many people are not comfortable with that. And so we also integrate with AWS Bedrock, which is Amazon’s hosted large language model service. And they have some security stuff around that so that those models are open source or hosted by AWS. And so if you can trust AWS, all your data lives there anyway. So maybe you’re a little more comfortable with the IP issues there. And then the third thing is Hugging Face.
00:19:25
So the open source models that are out there, you can actually instantiate those within a project of yours and use GPUs and so on in order to have that thing actually physically hosted in your environment so that you’re sending your prompts to yourself. And I guess the upside there is you can do some fine-tuning, you can really make those models yours in a significant way. So Zerve is designed to be self-hosted, so all your data and all your compute lives in your environment anyway. So you’ve got some choices around, okay, where can I send my prompts? Do I need… Is it low risk so I can use OpenAI? Am I comfortable with Bedrock or do I want to actually host the things on my own infrastructure?
Jon Krohn: 00:20:09
It’s nice that you offer that flexibility. And so it seems like that would suit kind of everyone. Is it more complicated for somebody to get set up if they’re running, say, Hugging Face models on-prem, or that’s kind of just as easy, just as turnkey as using OpenAI API?
Greg Michaelson: 00:20:24
Yeah, so if you can see it, then you can talk to it. So if you have it hosted somewhere else, it doesn’t have to be hosted in Zerve, although… Yeah, I guess it depends on the infrastructure. That’s complicated. Everything that we do is a serverless, so you don’t have long-running servers, and so it may be more cost-effective to operate that way. But that’s a complex conversation, so you might be running on a Kubernetes stack and you’ve got like existing hardware and stuff so it doesn’t… All that sort of stuff. So yeah, we work with each individual customer to figure out their situation in terms of where do they want their data to live, what compute do they want to use, which cloud provider, all that kind of stuff to figure out.
Jon Krohn: 00:21:06
Perfect. Sounds great. Okay. And then kind of my last question around features, new features in Zerve since you were last on the show is you talked about having an AI assistant. And so how does that… Is that kind of the same thing? Is this LLM that’s helping you generate code or is that something separate, a conversational kind of thing in natural language?
Greg Michaelson: 00:21:28
Yeah, it’s different. So we started… The first LLM integration that we built were what we call GenAI blocks. So in those blocks you’re not typing code, you’re typing in a prompt and those prompts can be dynamic. So you might take and connect a code block to a GenAI block and pass it some variables that you want to include in your prompt in order to get that sort of query back. And then you can take the output of that query and use it in downstream. So those are sort of internal. The AI assistant is above that.
00:21:59
So you might have a prompt, like a text field, an interaction, a chat space with a large language model where the AI assistant is actually an agent and it can do stuff to your canvas. So I might say, Hey, build me a canvas that does… That takes this data frame and you could reference a table in a Snowflake cluster or a Databricks setup or whatever, and write me an analysis that does whatever. And then it would actually create a plan around that and then actually it would be able to create the blocks and connect them and all that sort of stuff. So yeah, the AI assistant is really an agent that will actually do stuff for you. You can kind of direct it to edit different parts of your project or create new bits.
Jon Krohn: 00:22:48
That sounds-
Greg Michaelson: 00:22:49
It’s phenomenal.
Jon Krohn: 00:22:50
That sounds pretty damn cool. That sounds really cool.
00:22:50
So you could potentially have… So if you had like three people, three humans collaborating in real time on a Zerve canvas, you could have three people plus their three assistants, all kind of generating nodes and figuring out how do these flows work in some complex machine learning flow.
Greg Michaelson: 00:23:06
Yeah. Exactly.
Jon Krohn: 00:23:07
Nice. Okay, so this now starts to feel like we’re talking about a completely new era in data science. Like Zerve is at the forefront of building this new industrialized data science as opposed to a more artisanal data science where you have individuals working alone in a Jupyter notebook, typing out each character of code.
Greg Michaelson: 00:23:30
Locally.
Jon Krohn: 00:23:32
Exactly. And so this is now this kind of industrialization of data science that involves adopting platforms, machine learning ops systems and automation. What role does Zerve play in that? And obviously you have a ton of experience with automating things with your years of experience at DataRobot before that. And so fill us in on how you see this transformation happening, how organizations can best harness that transformation and be more successful on developing and deploying AI projects.
Greg Michaelson: 00:24:02
Yeah, we think about ourselves as kind of a full stack kind of data science environment. We don’t do the hardware and the data warehousing type stuff, but we connect to all of those solutions. But everything else we do. We’ve played with the term operating system for AI in terms of how do we talk about ourselves, but we really do all of the things. So you can connect to data, you can explore, you can create visualizations, you can publish reports. We integrate with a bunch of dashboarding things. I just did an AWS QuickSight dashboard for a conference that I was speaking at.
00:24:38
You can train models, you can use GPUs, and then when it comes to deploy, you can really easily build your own APIs that you can host within Zerve or download and take somewhere else. You can deploy using SageMaker, you can schedule jobs. The whole life cycle of these data science projects is built into one application and you can do it in any language you want. So you could have your data engineers writing SQL, you could have your old school statistician folks writing R code. You could have your machine learning engineers doing Python, and it all syncs to GitHub or Bitbucket or whatever source control you use. So that if you’ve got an engineer down the line and they want to stay in VS Code, it’s fine. It all pulls together on GitHub. So yeah, it’s awesome. It’s full stack.
Jon Krohn: 00:25:27
Very cool. Yeah. Any thoughts on my question there about how… I don’t know if you have any guidance for organizations who are trying to take advantage of this kind of shift. So it could be Zerve or maybe some other kind of tool that’s industrializing data science in some way. How can organizations be successfully adopting those, taking advantage of them to get more AI projects into the real world that are profitable?
Greg Michaelson: 00:25:55
Yeah, there’s definitely this… Businesses have always had to sort of juggle between build versus buy, right? There’s a million vendors out there that are doing a million different things. And some organizations are like, okay, we’ll go buy that. Others are like, okay, let’s staff a team that can build some of this stuff. And so you still have that sort of set of trade-offs when you go down that road.
00:26:19
But the large language model stuff has made it so much easier to build that. I think the calculation on build versus buy is super different now than it was a few years ago. It’s just so much… It’s going to be so much cheaper to build than it is to buy these sort of things. I actually think it’s sort of an existential crisis for a lot of these SaaS vendors that are building kind of custom-built use case type solutions and then charging a mint for them. Yeah. Who knows what the space is going to look like in a few years. I think it’s got ramifications for the VC space, certainly for the software as a service space. Yeah, it’ll be interesting to see what happens.
Jon Krohn: 00:26:58
Yeah, it is interesting. Everything is moving really quickly and so hopefully people are taking advantage of things like their favorite data science podcast to stay on top of all those kinds of things. You mentioned earlier how Zerve integrates with existing data stacks, and so this kind of flexibility, it seems like it’s kind of obvious to me, but maybe you can give us some specific examples maybe with existing clients that you have or other experience that you have in your career on why it’s important to be able to integrate with such a broad range of different kinds of data stacks. What are the main data stacks out there that you see people using and yeah, what’s the advantage of having flexibility across all those different stacks?
Greg Michaelson: 00:27:40
By data stacks you mean like warehousing type stuff? Like Snowflake, Databricks type?
Jon Krohn: 00:27:47
Yeah. Exactly, exactly.
Greg Michaelson: 00:27:48
Yeah. Well, I mean, it’s kind of a tail wagging the dog type situation, I think. There are a lot of considerations in terms of picking your applications that you use to store and interact with your data that are more than just like, okay, where does my data science happen? You’ve also got integrations with your finance systems and all the different applications that actually interact with your data. It’s not just data science. There’s way more that goes on there. So it’s not often that the data scientists get to decide what storage is used, particularly in larger organizations when they’ve been storing data for years and years. And so there’s entrenched sort of legacy systems where it’s stored. So newer companies tend to be using things like Databricks and Snowflake and things like that, whereas older companies might have… You might even see mainframes at a big bank or whatever it is. So our approach is just like people’s data is going to be where it is and you can access it via code. And so wherever it is, we want you to be able to get to it.
Jon Krohn: 00:28:56
This episode of SuperDataScience is brought to you by the Dell AI Factory with NVIDIA, delivering a comprehensive portfolio of AI technologies, validated and turnkey solutions, with expert services to help you achieve AI outcomes faster. Extend your enterprise with AI and GenAI at scale, powered by the broad Dell portfolio of AI infrastructure and services with NVIDIA industry-leading accelerated computing, it’s a full stack that includes GPUs and networking; as well as NVIDIA AI Enterprise software, NVIDIA Inference Microservices, models and agent blueprints. Visit www.Dell.com/superdatascience to learn more. That’s Dell.com/superdatascience.
00:29:42
Nice. All right. So let’s talk about an application that takes advantage of all those kinds of data stacks. You could have tons of information stored in whatever kind of database. And a really kind of buzzword technology today, which I would say was even buzzier in 2024, is this idea of RAG, retrieval augmented generation. But it is a really powerful thing and me and companies that I’ve worked for, we’ve had a lot of great experience leveraging RAG. So for example, as a concrete example that will make the value of RAG clear at a company that I co-founded, Nebula, they are a human resources platform that’s allowing you to search over all the professional profiles of everybody in the US in seconds.
00:30:27
And so you’re talking about hundreds of millions of professional profiles and each one of those professional profiles is a document filled with natural language. And so we can pre-compute vectors. So you can take a deep learning model of some kind, often today a large language model. And so you can take the natural language on each of those hundreds of millions of documents, encode it into a vector, so just a series of numbers. And so every single one of those hundreds of millions of documents gets encoded as a vector. And so you can imagine that you kind of end up with this huge table with say 100 million rows representing each of my 100 million profiles that I have in the database.
00:31:11
And then you have however many number of columns you think is important for your vector space. And so basically there’s a classic computer science trade-off of, you can double the number of columns, but then that’s going to double the amount of compute that’s required. So you kind of find the sweet spot for your particular application. And yeah, so you might have 64 or 128 or 3000 columns depending on your needs, but basically you end up with 100 million rows, something on the order of hundreds of thousands of columns representing the location of each of those hundred million documents in your high dimensional space in your hundred or your thousand or 3000 dimensional space is kind of the highest number I went to there in my little example.
00:31:58
So you pre-compute all those and then a user comes into the platform. In our case at Nebula it was, I’d like to hire a data scientist in New York, then we can convert that in real time in milliseconds into its own vector and find related documents across the hundred million profiles. Yeah, and then you can with today’s big context windows, you could take all of those documents potentially like the top hundred documents that come back, throw them all into the huge context window of a generative large language model. And then you can be, the generative LLM can be answering questions or pulling out information across those hundred documents. So hugely powerful technology. Definitely worth exploring for people.
00:32:42
There’s probably some listeners out there that are like, I know Jon, I know what RAG is. But for those who don’t, it’s definitely worth exploring and understanding. And so Zerve enables RAG to be scalable, so it distributes compute workloads automatically because there’s a lot of RAG that can be quite technical and difficult to get right. So yeah. How does Zerve’s approach to parallelizing these RAG workflows compare to if you tried to do that on your own, if you tried to figure out all the pieces of what I just described on your own?
Greg Michaelson: 00:33:12
Yeah. Every example is different. We worked with an organization that was doing media recommendation and it was something similar there. So you would want to go and type in, recommend for me sports comeback movies or whatever, and they would come back with, oh, you should watch Rocky, whatever it might be. But it turned out that naively querying these large language models would often give you results that weren’t very good. So like I recall one example, we fed it Dune, the movie Dune. I like Dune, give me more movies like that. And it ended up giving the top five responses. Four of them were other versions of Dune. So I was like, all right, that’s not ideal.
00:34:00
So we ended up doing something similar on the RAG front where we went out and we got top podcasts and New York Times reviews and we brought in all sorts of other documents to add into the context window, like you say, and we pulled it all back together into a recommendation that was then significantly better. And it included things like video games and podcasts and books and magazines and stuff like that. So more than just movies. So it became kind of a much livelier, much richer source of recommendations. So yeah, that RAG stuff is super, super convenient. Being able to lay it out in a canvas style, kind of like a graph as a dag, makes it really easy to kind of see what’s going on and it makes it really fast to sort of experiment with how it works.
00:34:53
And then to the extent like you’re in your example where you have a hundred million people processing all of that stuff, parallelization is key for a lot of different use cases. So one use case that we just worked on was trying to evaluate the performance of a call center based on audio files, audio recordings of like MP3 file recordings of conversations with customers. This was at a bank, that’s a lot of processing, right? Converting all of the recordings into text that you could then pass to a large language model. You need to be able to do parallelization for stuff like that. Otherwise, it’s going to take… It’s not feasible to do it. So the parallelization turns out to be key for a lot of these use cases, but it doesn’t have to be just large language model stuff, you know, it could be heavy compute loads of other types. We’re agnostic in terms of what loads are actually getting parallelized. We just make it easy to do it.
Jon Krohn: 00:35:54
All right. So yeah, that’s cool, Greg. And so I’ve got another great soundbite for you here. So in addition to your code nodes, did you realize that you have RAG DAGs?
Greg Michaelson: 00:36:07
Oh, I don’t hate that. That is definitely going to be reused.
Jon Krohn: 00:36:11
Yeah. And so it sounds to me like the key, whether it’s a RAG DAG or some other kind of high compute load DAG, one of the key wonderful things that Zerve is doing is distributing those workloads automatically, which is cool. Nice. So another kind of tricky thing that data scientists maybe even myself have difficulty with is deploying AI models. So something that’s been intuitive for me for literally decades is opening up some kind of IDE, Jupyter Notebook, something like that, and getting going on inputting some data, doing some EA and building a model.
00:36:54
But the thing that hasn’t been intuitive to me, and it’s probably just because I haven’t been doing it as much, I’ve had the luxury kind of working at companies where machine learning engineers or software developers, backend engineers take and then the model weights that I’ve created and they put them into a production system. So on a smaller team or on a team where there’s huge demand for software engineers, which is often the case, you can end up having more data scientists creating models than there are software engineers to deploy in a lot of companies, that creates a bottleneck. So how Zerve’s built-in API builder and GPU manager remove those kinds of barriers?
Greg Michaelson: 00:37:33
Yeah, it’s not just a bottleneck, it’s also kind of a problematic dependency because at the end of the day, the software developers that are deploying these things probably aren’t data scientists. So it’s not obvious that they are going to understand what is supposed to be done. And there’s a lot of subtlety to this sort of thing. So you can get mistakes introduced in really easily here as well. So yeah, so if you think about the deployment process, there’s a lot of hurdles to overcome. If you’ve ever been slacked or emailed a Jupyter Notebook and tried to run it, you know what some of them are, right? You have the wrong version of this package installed. Oh, you got to PIP install a whole bunch of other stuff to make that work. And so you might spend an hour trying to even get the code to run, assuming that you have the data and that all the folders and file paths are the same and all that sort of stuff.
00:38:33
So at the end of the day, what data scientists spend most of their time doing today is building prototypes. And then those prototypes get handed off to another team to recode in another environment with [inaudible] and deployed and managing servers and stuff like that. But it’s not obvious to me that data scientists know how to do that and it’s really not obvious that they have the privileges to do those kinds of things in terms of just the infrastructure and all that kind of stuff. So Zerve kind of handles all of those problems. So every canvas inside Zerve has a Docker container that’s supporting it. So anybody that logs into that canvas doesn’t have to worry about dependencies because it’s all saved in that project. And so those environments are reusable and shareable and so on.
00:39:22
So if I wanted to start a new project using the same docker container that another project was in, it’s really easy to do that. And so when you have a new data scientist join your team, they don’t have to spend their first week getting Python installed and making sure everything, oh, we use NumPy 0.19 and you’ve got 0.23 installed and none of those conversations have to really happen anymore as we manage all of that. And then let’s say that I did train like a random forest. I mean, you mentioned using your weights, like if I train a linear model or a logistic regression or something, then maybe it’s just a vector of weights that need to be handed off.
00:40:01
But if it’s a more complicated model like a random forest or an XG boost or a neural network or something like that, it’s not as simple as just like here’s some weights to put into a formula. It’s a more complex thing. And so then you’ve got to figure out, okay, I’m going to serialize this model, pickle it and then dump all the dependencies out and dockerize it and then hand that thing off, that’s also beyond the skillset of a lot of data scientists too. So Zerve handles all of that. So every block inside of Zerve, when you execute it, it creates serialized versions of all of the variables that you’ve worked through. So if I train a random forest in a model or in a block, then it’s there and it’s accessible. So I can access it from external to Zerve using an API, I can reference it in other layers.
00:40:52
So when it comes time to say make an API, maybe I want to make a post route where I send in a payload of predictor columns and then I want a prediction back from that random forest, well, then I just say, “Hey, remember that random forest?” And I just point at it instead of having to figure out how to package that thing up so that it could be deployed as an API. So we handle all of that stuff and then when you deploy in Zerve, you also don’t have to worry about the infrastructure stuff because all of our APIs utilize Lambdas. Like serverless technology, again, so you don’t have long-running services that are out there, it’s just there.
00:41:31
So a lot of the infrastructure stuff and the DevOps stuff and the kind of picky engineering stuff that can trip you up is stuff that we’ve just sort of handled so that it’s easy for the user. And that means that data scientists can start to deploy their own stuff, but in some organizations they may still might not be allowed. So then we have like a handoff system where it’s really easy to take something that a data scientist has done who by the way aren’t building prototypes anymore. Now they’re building software that can actually be deployed in Zerve and we can hand that off to other teams to actually do the deployments.
Jon Krohn: 00:42:10
Awesome. That does sound like something that would be useful to me. And I’m sure a lot of data scientists out there helping me get my models into production and feel confident about what they are. I like how you kind of turned that on its head as well and made me feel good a bit about myself, about the skills that I do have and a software engineer might not in terms of understanding the model that I’ve built. So I really appreciate that. I think that’s the end of my questions that are directly related to Zerve in any way. But as I transition more broadly, how can somebody get started with Zerve today? How can a listener who’s heard all these great features that Zerve offers this completely new way of working on data science, how can they pick that up and get going today?
Greg Michaelson: 00:42:51
Well, we’ve got a free tier that folks can get in and utilize and so you have all the flexibility and stuff. There are some caps with respect to like compute and stuff like that. We originally had a free tier, but we had to shut it off for a little bit because of the Bitcoin miners. They went a little bananas. So we had to turn that off and build some controls and stuff like that. But the free tier is back and so anybody can get on there and get in and give it a go.
Jon Krohn: 00:43:19
Those damn Bitcoin miners, what won’t they ruin? And so yeah, now moving beyond Zerve’s specific… Questions that could really be directly related to Zerve in any way. You mentioned earlier in this episode how AI could kill, how large language models could kill a lot of SaaS software as a service businesses. So some companies like Klarna are combining AI standardization and simplification to shut down SaaS providers, specifically they said. I mean, they have a specific quote in inc.com that I’ll be sure to have in the show notes about that. Similarly, Microsoft CEO Satya Nadella predicts that business logic will move from SaaS applications into AI agents.
00:44:08
And so it sounds like you’re in that same kind of boat as Klarna and Satya Nadella around thinking that with LLMs empowering them, internal processes can be simplified, standardized, and lots of different SaaS vendors can be removed, cutting down costs. There are other views out there that I want to highlight just quickly. So for example, the CEO of Zoho argues that in contrast AI will fuel new vertical SaaS companies because you get new problems to tackle. So we’d love to hear more about what you think of this and maybe which kinds of SaaS businesses should be most concerned.
Greg Michaelson: 00:44:48
Yeah, I think this goes back to the whole build versus buy thing. It’s just becoming so much easier to build stuff in-house. So nobody’s going to ever go out and go, okay, let’s… Like a CRM system, like Salesforce. There’s no risk that somebody’s going to go, okay, I’m just going to rebuild Salesforce from scratch. Although as a user of Salesforce, I say that with some regret. So there are big, big complex things that exist and it’s not likely that an organization is going to rebuild that sort of stuff. But there are an awful lot of vendors out there that do things that could easily be built by an organization if they wanted to do it in-house. So I think the newer stuff, the more cutting-edge stuff, every SaaS company seems to be bolting on generative AI onto their products.
00:45:43
But I think there’s a difference between a simple bolt-on where you add a prompt, a place where you can write a prompt or something like that, that’s kind of like a dime a dozen as compared to a SaaS offering that is kind of deeply integrated with these kinds of things that sometimes actually seem magical in some sorts of ways when the agents begin doing things that are just sort of intuitive and just sort of works. So yeah, it’s hard to say what’s going to happen, but I just see it being so much easier for organizations to build their own stuff these days. And it seems to me like that’s going to be a real problem for a lot of software vendors.
Jon Krohn: 00:46:27
Hey hey! This is your host, Jon Krohn. I’m excited to announce that this northern-hemisphere spring, I’m launching my own data science consultancy, a firm called “Y Carrot”. (If you’re an ML practitioner who’s familiar with “y hat”, you may get our name!). Regardless of who you are, if you’re looking for a team that combines decades of commercial experience in software development and machine learning with internationally-recognized expertise in all the cutting-edge approaches, including GenAI, multi-agent systems, and RAG, you’ve found us! We have rich experience across the entire project lifecycle, from problem scoping and proof-of-concept through to high-volume production deployments. If you’d like to be one of our first clients, head to ycarrot.com and click on “Partner with us” to tell us how we can help! Again, that’s Y Carrot, y-c-a-r-r-o-t-dot-com.
00:47:17
It seems like part of what’s so tricky for those SaaS vendors is that instead of wanting to pay a monthly fee that’s just kind of this flat fee, instead, we’ve kind of gotten used to now with calling APIs and paying per number of tokens that we send or receive back from that API. We as developers of AI products and decision makers within software businesses or I guess businesses in general, we’re getting more and more used to this idea of an economic model where it’s consumption based on specifically what I need instead of subscription based. I’ll give you an example that’s kind of interesting.
00:47:55
So a friend of mine, Shaan Khosla, brilliant software developer, he has built a really simple user interface that looks exactly like ChatGPT or pretty close. It has the same look and feel as ChatGPT, but instead of paying a $20 a month subscription for ChatGPT Plus, he uses that interface, simple interface, and he calls opening AI models on the backend and he’s like, my cost of using OpenAI APIs went from $20 a month to $2 a month. And yeah, so it’s kind of an interesting thing there. I’ll let you speak now. I’ve been speaking for way too long.
Greg Michaelson: 00:48:36
Yeah, I think I’ve used something like that as a product called TypingMind, I think is the name of it, that is doing the same stuff you put in an OpenAI key and then you might spend 20 cents a day or something like that. Of course I’m on the $200 a month plan for OpenAI.
Jon Krohn: 00:48:51
Me too, me too.
00:48:52
It’s so good. It’s crazy. I mean, I can understand for listeners, I did an episode entirely on this recently, so that episode number was number 870. And so I make the case in that episode and it sounds like you agree 100%. Yeah, I know it’s expensive. $200 a month sounds like a lot. But as soon as you start using it, there’s so many everyday use cases where a single report that it creates for me, I’m like, that is worth $200 to me and I can do it 100 times a month. So it’s a no-brainer.
Greg Michaelson: 00:49:25
Yeah, totally. I was on there yesterday and I was looking at… I was having a review and shorten a document to do like a first pass of it and it opened up this canvas mode, which was interesting. I couldn’t figure out how to actually use it, but it looked like you could edit the document in real time while it was editing it and work together somehow. I didn’t have time to kind of explore it, but yeah, they’re putting out new stuff all the time. It’s cool.
Jon Krohn: 00:49:49
Yeah, they are. But yeah, I think I kind of interrupted you. You were talking about TypingMind.
Greg Michaelson: 00:49:53
It’s just a product that… I think that’s the name of it. I could be wrong, but yeah, it’s just another-
Jon Krohn: 00:50:01
I’ll try to find it and include it in the show notes. But it is kind of interesting though. So for me as well, like Shaan Khosla who I was just describing, he was like, “Why don’t you just use this? I’ll just give you another login.”
00:50:13
He’s like, “It costs me like nothing for you to be doing here.”
00:50:16
But I was like, “Eh, but can it do deep research like this?”
00:50:19
And he’s like, “No, can’t do that.”
00:50:25
So yeah, so I think it’s definitely worth it. It is highly interactive, like you say, I haven’t seen that canvas yet. But the level of interactivity, the way that it asks me before it goes off and does a long research request for clarification on some of the key points, even just those questions that it asks, I’m like, those are incisive and exactly spot on the questions you should be asking as a well-educated analyst or PhD student or PhD graduate being asked to take on this task. All right, so in addition to the broad topics that we’ve already covered already, another big topic related to some of the tools that we’ve just been talking about, like LLMs, which you both incorporated into Zerve, as well as we use in our professional and personal lives through things like ChatGPT Pro subscription and deep research. Billions and billions of dollars have been spent on developing these LLMs and aggregated across the planet, LLM-based services and projects.
00:51:26
But there are still some shortcomings today around bias and inaccuracy and the LLMs, I start to notice it so less frequently that I start to just trust the AI systems, which is maybe risky. So when they were frequently inaccurate or having biases, I was like, you were constantly had your… You were constantly on the lookout for those kinds of issues. But now that I rarely see those, especially something like deep research or an o1 or an o3-mini or a DeepSeek-R1 kind of model that can iterate over its responses and fact check, it just seems like they’re basically always right. To me, that’s maybe a different issue.
Greg Michaelson: 00:52:17
Well, I mean, it’s funny though, I was… I don’t know, maybe six months ago, I was trying to write some code that would do… I wanted to use the API for the OpenAI API to submit an image and have it edit the image, which is not something that they could do then. In fact, I don’t know that they do that now even, I don’t do much image stuff. But I asked ChatGPT how for some code to do that and it invented an API out of nothing that didn’t exist.
00:52:48
And it was like, oh, here’s the code to do that. And I have no doubt that when they do introduce that feature, that the code would run. But no, it was just made up. It just completely hallucinated because they want to be so helpful. But the one thing I’ve found about these models is that if they get it right or at least close to right the first time, you’re probably okay. But there are some problems that I’ve given them and it’s like, all right, it didn’t get that right at all. And then I’m like, okay, there’s no hope, you take another route or something because if they don’t get it then they’re not in my experience anyway, easy to correct.
Jon Krohn: 00:53:26
Yeah, that makes perfect sense. Speaking of your inventing an API to call, similarly, I had the experience actually with deep research recently where I was trying to use the o1 model and at least at the time that I was doing this a week or two ago, it turns out o1 doesn’t have internet access yet, or was it o1 Pro? It was o1 Pro specifically that I was trying to use. And so I was like, I want to use this as a big problem. I want to get the chunkiest model that OpenAI can give me running in the backend in my deep research. And it turns out that they, at least at the time that I ran this that didn’t yet have internet access.
00:54:08
So if I use something like o3-mini high, I could get internet access, but not with o1 Pro. And so I didn’t know that though. And so I provided a link. I said, go to this link, summarize the information, or something like that. And luckily, I at least have that trace, when you do deep research, it’s kind of an explanation of what it’s thinking about. And so in what it’s thinking about, it was like, because I don’t have internet access, I’m just going to assume what kind of thing was there. And I’m like, no. So at least I could see that it had that trace because otherwise with the output itself, you’re kind of like, oh, cool, that makes sense.
00:54:47
And so if I hadn’t gone and looked at the website or looked at the trace, I would have been led completely astray. So yeah, so interesting. These kinds of issues that we do still have. Yeah, I don’t know. Do you have any insights yourself on how we could potentially be navigating this thorny situation? So as we increasingly automate judgment and decision-making, how should we balance these kinds of biases that happen with both humans and AI evaluation processes to create more fair, transparent systems?
Greg Michaelson: 00:55:16
Yeah, I’ve done a lot of the iterative stuff. Like one kind of fun project that I was interested in is I wanted to figure out if the models were biased towards their own answers. Like if they thought that their own answers were better than the answers coming out of other large language models. And so I built a project where I asked for 100 SAT essay questions. Give me a list of essay questions, and then I passed those essay questions to the large language models and asked them to answer them. So hundreds of answers coming from, I think I used four different large language models.
00:55:54
And then I asked those same four large language models to grade the sets of four, rank them whose answer was the best. And so hundreds more requests there. And then I sort of plotted it and looked to see, does the GPT like Titan or did… Whatever, right? Evaluating them. It was really interesting and wildly unstable. I could rerun that analysis and it would dramatically change, but there didn’t seem to be a bias there in terms of like ChatGPT likes its own answers, that sort of thing. But in terms of… I found it really effective to use LLMs to evaluate the answers from LLMs. So that whole iterative thing that they’re now sort of doing under the hood kind of explicitly doing that.
00:56:42
I heard a talk at a conference last week, the AWS conference in Ireland. They were talking about distilled models where you have a teacher model and a student model and you’re kind of sort of like a reinforcement learning kind of thing that they get going on there. I don’t know the specifics of it, I haven’t looked into it, but that was another thing that was interesting. So kind of playing the models off one another, a good cop, bad cop kind of situation, or like maybe you have a cheap model and an expensive model and you sort of use cheap models to evaluate your answers and you only call the expensive models when the answers look a little questionable, that sort of thing. So yeah, so it’s LLM committees I guess is the way to go.
Jon Krohn: 00:57:28
Nicely answered as I asked that I was like, this is maybe a really out there question that Greg might not have any ideas on. And you nailed that. That was a really great answer. I think that’s absolutely right, an LLM committee, and depending on your particular use case, there’s clever ways of potentially doing it. Like you said there of having a cheap LLM for most use cases, but then another cheap LLM that’s evaluating that and bringing in a more expensive element to handle those trickier situations.
Greg Michaelson: 00:57:56
Yeah, I think there’s a lot of research that needs to be done to figure out what the most effective committee structures are. But I think thinking about it like a committee is actually a really interesting kind of metaphor for figuring it out.
Jon Krohn: 00:58:06
Ask your three favorite LLMs how to tackle the problem. So this is an exciting time, obviously to be a data scientist now that we have all these kinds of tools. Tools like Zerve, that allow us to deploy models to parallelize models easier than ever before to collaborate on data science workflows much more easily than ever before. Without going into your proprietary plans, I’m not asking for you to go into that, I’m just… Kind of generally speaking, you must spend time, Greg, thinking about where the future is going with all this, like where do you see in five years, 10 years, 20 years, what is a data scientist or a software developer’s workflow look like?
Greg Michaelson: 00:58:48
Yeah, it’s so crazy. I’m not looking five years ahead, I’m looking one year ahead at this point, it’s just moving so fast. No five-year prediction is going to be reliable the way things are going. We’re trying to stay flexible because these models are just getting better and better. I do think large language models are kind of the… They’re going to be the kind of core of how things move forward in the engineering space, in the data science space. And so I think the most important priority for us right now is to build the best AI assisted coding experience that you can so that it becomes very streamlined and very easy for a user to just type what they want and then they get it.
00:59:29
And so that’s kind of the path that we’re on. We’re very focused on the data space, so we’re like, we’re not looking at JavaScript or any of the other… Because there’s lots of programming languages out there and you could kind of do the same thing there. But we’re very focused on the data space because I think that’s where… Well, I mean that’s the audience that we’re working with, code first data users, so. But yeah, we want to build the best AI powered coding environment that you possibly can.
Jon Krohn: 01:00:01
Nice. That sounds great. Very exciting future ahead, no question about that. And fast moving, it makes sense that you would not try to make a speculation about a five-year or a 10-year plan.
Greg Michaelson: 01:00:16
Assuming we’re not all on universal basic income and just out painting pictures in the park-
Jon Krohn: 01:00:22
It’s the 10-year prediction.
Greg Michaelson: 01:00:23
Playing fetch with our dogs.
Jon Krohn: 01:00:25
Yeah, maybe the five-year prediction. So, very cool. I guess we might check in with you again soon. It’s always so much fun having you on the air. I had a lot of laughs today, I really enjoyed this. So maybe we’ll be checking in with you again in the future, in the near future and we’ll see how your predictions have played out. In the meantime, we do need another book recommendation from you, if you may.
Greg Michaelson: 01:00:51
Fiction or nonfiction?
Jon Krohn: 01:00:54
Whatever comes to mind first.
Greg Michaelson: 01:00:56
I just finished a book called Mickey7 and it’s funny because I saw it on my recommended list and it must’ve been there because it’s about to come out as a movie, but Robert Pattinson is supposed to play the main character in this book and it hasn’t come out yet, but I really enjoyed it. It’s about a guy, they build the technology to imprint your consciousness and save it and so on a dangerous mission, they have what are called expendables, which are people that you… They’re basically immortal, but they’re not immortal because you just reprint their bodies every time they die. So it’s sci-fi, it was an entertaining read. A little light-hearted I guess.
Jon Krohn: 01:01:39
Nice. Sounds interesting. All right, and then for our interested listeners who enjoyed your good humor and your insights and insights on the field of data science, how should they follow you after this episode?
Greg Michaelson: 01:01:49
Well, I’m on LinkedIn. I haven’t been doing much TikTok stuff lately. Not since my cereal… I sold my cereal business. But yeah, I’m on LinkedIn, you can always see me there. Try to post interesting projects, interesting stuff from time to time. And of course, Zerve.
Jon Krohn: 01:02:08
Yeah, yeah. And so we’ll have… There’s probably other Greg Michaelson’s out there, so we will disambiguate by having a link specifically to the Greg Michaelson at Zerve in the show notes on LinkedIn, although you can also just type that, that seems to work pretty well in LinkedIn for disambiguating. Greg Michelson, Zerve. Awesome, Greg, thank you so much for being on the show and yeah, as I said, hopefully we’ll be checking in with you again soon. Really enjoyed this today.
Greg Michaelson: 01:02:35
Always a pleasure. Thanks for having me.
Jon Krohn: 01:02:46
What a mensch Dr. Greg Michaelson is. In today’s episode, he covered how Zerve organizes code in a directed acyclic graph, DAG, where nodes contain code, code nodes, and edges show data flow enabling real-time collaboration across Python, SQL and R. He talked about their new Fleet feature and how it parallelizes code execution using serverless technology, dramatically reducing processing time for tasks like LLM calls without requiring additional code. And he talked about how Zerve’s AI assistant can now act as an agent that creates entire project workflows based on natural language instructions.
01:03:20
Greg predicts that with LLMs making custom development increasingly accessible, many SaaS businesses may face an existential threat as companies find it cheaper to build rather than buy solutions. And when working with LLMs, Greg recommends using committee approaches where multiple models evaluate each other’s outputs to reduce bias and improve accuracy. As always, you can get all the show notes including the transcript for this episode, the video recording, any materials mentioned on the show, the URLs for Greg’s social media profiles, as well as my own at www.superdatascience.com/879.
01:03:55
And next month, if you want to meet with me in person instead of just on social media, I would love to meet you in real life at the Open Data Science Conference, ODSC East, which is running from May 13th to 15th in Boston. I’ll be hosting the keynote sessions and along with my longtime friend and colleague, the extraordinary Ed Donner, you’ve got to see him speak if you haven’t already, seriously. We’ll be delivering a four-hour hands-on training in Python to demonstrate how you can design, train, and deploy cutting edge multi-agent AI systems for real life applications.
01:04:30
All right, thanks to everyone on the SuperDataScience Podcast team, our podcast manager Sonja Brajovic, our media editor Mario Pombo, our partnerships manager, Natalie Ziajski, our researcher Serg Masís, our writer Dr. Zara Karschay, and our founder Kirill Eremenko. Thanks to all of them for producing another entertaining and educational episode for us today, for enabling that super team to create this free podcast for you. We are deeply grateful to our sponsors. You can support this show listener by checking out our sponsors links. Give them a click. They are in the show notes. And if you’d ever like to sponsor an episode yourself, you can go to johkrohn.com/podcast to get some information on how to do that.
01:05:13
Otherwise, support us by sharing the episode with someone who would love to listen to it or view it. Review the episode on your favorite podcasting platform or YouTube. Subscribe if you’re not a subscriber. Edit videos into shorts, whatever you want. But most importantly, I just hope you’ll keep on tuning in. I’m so grateful to have you listening and I hope I can continue to make episodes you love for years and years to come. Till next time, keep on rocking it out there and I’m looking forward to enjoying another round of the SuperDataScience Podcast with you very soon.
