Private AI with Dell Technologies

Machine-generated from the episode audio and not hand-corrected, so names and technical terms may be imperfect. The audio is authoritative.

Alright, you're watching the kickoff or listening to the kickoff of coverage of Dell Tech World 2024. You know, Michael Dell, and I don't know if you can catch it behind me if you're watching this on video, we've had a LinkedIn post proud that Dell Tech World for the first time is a pretty big AI conference. I'm going to challenge that statement a little bit. I got two of the perfect folks from Dell Technologies to talk to me about Dell's efforts in AI.

Nathan Sharer, welcome to the podcast. Yeah, thanks for having us, Keith. So, Sharer, talk to me about what what are you folks doing with AI within Dell itself? Well, that's a great question to ask. And I might start by asking, you know, what aren't we doing with AI these days? I think you're gonna laugh at me when I say that. You know, there's a lot. And it's I think that there's a couple of functions. I'll let my friend Nick here talk a little bit about what we're doing externally with customers.

I want to start by talking a little bit about what we're doing internally inside of the company. And there's a lot of transformation at hand. And I think one of the great things about what we're doing is it's a little bit of a microcosm for what I think customers are also grappling with, which is how do you transform, you know, a Fortune 50 company to embrace these technologies as they're emerging at, you know, warp speed. And so a lot of what we've been doing is sort of, you know, staying abreast on the technologies, you know, as fast as they're coming.

We've been implementing them. We've been, you know, I've been, you know, spending a lot of time, you know, personally running trainings and helping people learn how to, you know, embrace prompt engineering and bringing, you know, these technologies into their day-to-day workflows. You know, I think training and education is such a core part of this, you know, as we're watching these technologies emerge, it's, you know, as important as, you know, understanding how to architect these systems and, you know, the fundamentals of, you know, the infrastructure that we, you know, we spend so much time, you know, talking about.

There's also a huge onus on us as tech companies to help individual users inside of companies understand how to use them at a rudimentary level, you know, and so that's a lot of, you know, what I... Yeah, and I thought it was really funny because you mentioned all of that and then we didn't even get into the Dell Digital stuff. Right. Where, you know, we have a team of folks that, you know, are really, they've been on this long journey, transformative journey of modernizing all of our technology.

And what's really fascinating is sometimes that you go out, you do these things ahead of time and then something like this new technology, like generative AI pops up and now you've got, you know, a CI-CD pipeline, you've got all of the private cloud architecture that's foundational and now you can then, you know, make these models available and play around with them and make sure data scientists have access to the right functionality and tooling. And, you know, we did a lot of experimentation last year.

I think they said something like there was 800 or 900 different projects. Everyone gets excited, all of the business units, that they see the potential and they start moving towards, you know, playing around with these technologies. And what we really realized was that we had to ground our efforts though. We had to narrow our focus. And this is something I hear from all of the customers I talk with, right? Everyone's like, we have 100, 200, 300 use cases. And so the trick is how do you start with just a couple?

And so our services organization actually partnered with Dell Digital and they went through an entire process of figuring out which of these use cases would deliver the highest ROI and were most feasible. And we really have narrowed it down and we started there and we've been working through there. So, you know, that's great because we're getting the tool sets for ourselves. Now, you know, when Sharon had said, you know, where aren't we doing this? You know, there's a whole slew of other things that we're doing too, where it's like, you know, there's a lot of AI ops types of technologies that we have, you know, CloudIQ and OpenManage Enterprise.

And so we're building AI into a lot of our products as well. And then comes the area that I spend a lot of my time on, which is we have these platforms, which are fantastic platforms to be the infrastructure for your generative AI efforts or customers generative AI efforts. And that's really my focus. That's where I'm really interested is telling that story, helping people understand the value across the entire portfolio. Because we have something that, I mean, maybe there's one or two other companies on earth that potentially could say this, but like we have this broad portfolio, everything from PCs to servers to cloud technologies, that's really hard to assemble for anyone else.

So I had a, did a project with you earlier this year, Nick, and that's what prompted this interview where we took a Dell Precision workstation and we installed a local copy of, I think it was the model was Llamatune, did some rag and did some pretty impressive stuff. That was on a laptop. And what I've heard you call out is the work that Dell has done in private cloud over the past few years, actually, and advancing the art of being able to bring data on-premises.

I think one of the things that my audience will push back on is that private cloud is probably, or hybrid cloud isn't the model for hosting just AI at scale. What are, as you're talking, either one of you are going out and talking to customers, how's Dell Technologies helping to address kind of the same challenges we had in private and hybrid cloud when it came to data privacy, sovereign cloud is now a huge issue all over the world, and just the far edge when it comes to AI.

Yeah, I think the first thing is we've seen people's preferences swing back quite dramatically for this workload. And I think that's what you have to think about. You have to think about AI as a workload, and there's some unique characteristics that you have to think through. And that's where people can get into trouble with public cloud is, when I think of it, if I was to tell people my advice for when to use public cloud, I would suggest that they use GPU as a service versus AI as a service.

And what I mean by that is there's these cool models that people have developed, and you can consume them as a service, but that's closer to SaaS than infrastructure as a service. And the problem with that is when you're talking about SaaS, that means that the vendor has to hold the keys to the data. And that's where I'm like, I'm out. Because when they hold the keys, and they'll have very good, very detailed security posture stuff on their website, so you can go down and look at the fine print.

But the reality is there is a non-zero chance that a human being outside of my company could potentially see data that I have in there or queries that I've put in the model. And it's not that I don't trust the vendor, but there's governmental entities that could get involved. There's potential for hackers or bad actors. And the problem is when you get to SaaS versus IaaS, it's really hard to monitor who has access. We've heard about some of these things in the news about email getting hacked and not finding out for a few months and things of that nature.

And so that would be my concern is when I have it in my own data center, I have the most control possible. And I think that's one of the unique benefits of the approach is we're helping organizations bring AI to their data instead of asking them to take their data and put it somewhere else where maybe they have less control or maybe now there's some new data sovereignty challenges. So one of the things that I also wanted to talk to you both about is this relationship with NVIDIA.

Jensen, we're at GTC. Jensen goes to the Dell booth and says, this is the company to buy GPUs through. But Dell is much more than a hardware company. And NVIDIA is much more than a hardware company. I've heard this term AI factory thrown around a few times. Can either one of you kind of explain this relationship and the concept of the AI factory? Yeah, I can start that and then Sharon, maybe you follow up and about some of the value of it and what people are seeing from the actual outputs.

The idea of an AI factory really comes down to having your technology be informed by the business. And to your point, I think one of the things that really stood out and wowed people at GTC was our services. They saw that our services frameworks, they saw how we were approaching this and they could buy into it. And it was really cool because we had like these posters and we had like these little maps of your as is and your to be and all of the things along the way.

And that's part of our services engagement if you engage services for strategy. So when you think about the AI factory, it always starts with the outcome. Every single conversation that I'm hearing, whenever I sit down, whenever I listen to our execs, we're always leading with the outcome. So what's the use case? What's the role that we need AI to play within the company? From there, we need to then go to the data and we need to say, what's the status of your data?

Where is it at? Is it protected? Is it prepared? Is it labeled? And what we're seeing is that, and I've been waiting for an opportunity to say this because you're a cowboy hat, is it's the all hat, no data is what people are running into. I was like, I was waiting for it. But that's the challenge. The data is what's really going to differentiate things. 7 trillion parameter model. If you have a use case, it's really well-defined and you've got great data.

And so from there, then we get into figuring out what's that infrastructure look like. Is it AI PC for a knowledge worker and you're just deploying something like co-pilot on it, or is it rag or fine tuning and maybe now I do the next C9680. Um, we have the breadth of that, um, that infrastructure. And so we, we, we, we grab that right infrastructure. And then we move up to our ecosystem and we say, you know, what, what's the models that we want or, or the customer want, what, what kind of accelerators are available and, and, and kind of, you know, what is the frameworks and the tools that we can help land for them.

And then we round that off with the services. The services is really what completes the whole thing. But you mentioned Nvidia, you know, I think of the work we've done with Nvidia, maybe they're the best example of, you know, what happens when this AI factory comes together, right? Because you're getting a full stack. It's a Dell validated design. It's got 340,000 hours that have been put into validating that design and helping with the build out of it. That's all that work that your customers or our customers don't have to go and do.

Right. So that's pretty amazing. Sharon, what do you think about the outcomes here and, and, and with the AI factory? Any, any thoughts on that too? Yeah. I mean, I think the most important thing is a lot of organizations don't necessarily know how to start. They know where they want to go. And a lot of times they're looking to get there as fast as they can and also do it in a way that maximizes value and efficiency. Right.

And so I think sometimes what we need to remember is that, you know, sometimes it, you know, it gets very complicated and, you know, it's not always about the largest model. It's not always about, you know, the most fanciest, you know, infrastructure, the most, you know, sometimes it's about the right solution with the right size model matched with the right use case. And I think that's what we're trying to do. So I guess I'll end this up with the last question that I get a lot from customers who have invested in AI infrastructure.

They can't find the use case outside of chat. They've kind of fallen over a little bit. Sharon, I'd love to hear thoughts from you. What types of use cases have really moved the needle for Dell customers? Yeah. Well, I mean, I think what types of use cases and move the needle for Dell customers is an interesting question because I think use cases right now are the big question for everyone. Right. And I think there are a number of use cases that we have worked to identify that we think are going to move the needle just, you know, broadly.

And I think a lot of it right now is a lot of organizations are, you know, in early days implementing them. Right. So the use cases that we have been focusing on that we've been implementing internally ourselves, that we have been advocating for other folks to implement are around content creation, they're around document automation, they're around, you know, what we call synthetic data or data design. They're around digital assistance, they're around natural language search. So these are all areas that we sort of identified based on, you know, conversations we've had in our Accelerator workshops with other customers.

They're also, you know, we talked a little bit about the 800 use cases that we identified in our own organization in our service engagement. And this was, you know, something that came about looking at, you know, these use cases internally and realizing that when you look at these use cases, they could look very different. But when you sort of took a step back, kind of realized there were some commonalities, right. And that kind of helped us understand that, you know, there was a way to sort of cluster and prioritize them and say, hey, these are the areas where we really think there was a lot of opportunity for exploration.

And so, you know, as we've been sort of going out and meeting with customers, we've been saying, hey, you know, these are the places where we really think if you, you know, put a little bit of muscle behind it, there's a lot of opportunity to, you know, to derive a lot of value. Yeah. And I think, you know, it's really fascinating for me is I think what people are struggling with is the manifestation of the use case, right. Where it's like, if you play around with these tools and you're using them in your own little world, it's very easy to get value out of them.

Like I've been able to write a blog in 20 minutes using these tools. I like to think of it as Malcolm Gladwell had the 10,000 hours to be an expert. And I think that this AI kills that because I can be an expert in something very quickly. I can learn super fast, but by, you know, interrogating the data and asking questions and using document automation, all this, all these types of things, the challenge with that is scaling it across the organization, because I'm really good at it.

Sharon's, she's the rockstar, you know, she, five minutes later, after we make a joke about something, she comes up with the AI generated song for it, you know, so she, she knows all the tools and she, she, she does all the different ways to use them. The challenge is how do I get the rest of my workers to use them? And how do I get them to get the value? And I think the thing is, is that it's more than just deploying a technology and saying, well, there you go.

And we're done here. It's like, you need to do the onboarding. You need to help with the prompt engineering exercises. Sharon's had a newsletter inside Dell. I think it's been over for over a year now. And, you know, people are opting in all the time, all kinds of people, because they want to learn how to use this technology. And there's no better way to do that than the constant stream of these reminders of all the different ways it could work.

Yeah. I'm always interested in how many new ways I'm learning to use chat LLMs. And I'll, you know, I'll type in a prompt, I'll get a response. And I'm like, oh, I don't like that part of the response. And someone reminded me, you know, someone like a Sharon reminded me, well, you can just tell the model to not give you that part of the answer. You know, just simply say, you know, just give me this part of the output and forget the rest of the fluff.

And I've worked, you know, in this, this tuning it with my own data and putting my own blog post histories, uploading blog posts at the blog posts that the blog posts for it to get my voice when I want, when I'm missing that paragraph, uh, that piece that I'm like, oh, you know what, this is missing something. And what I'm learning about LLMs is that they're fabulous at understanding the human language, the communication, and that's the key. It may not be an expert in what I'm an expert in, but it is an expert in communication and leveraging that.

So Sharon, I would love to uh, learn more about some of the work that you're doing internally at Dell, helping to accelerate teams. Nick, people can find you everywhere. You're doing these workshops, like seems like old. Yeah. You know, and I wanted to, you just reminded me of something that happened in one of the workshops. So, uh, Ken DeRosa was in our workshop and he, he's, uh, you know, uh, what was he, uh, the chief, uh, CTO, um, the opposite of CTO.

And he, he, he went out there and he said to everyone, he said, how many people are on earth as you know, six, seven, 8 billion, you know, whatever, you know, whatever the numbers up to. And he goes, now I want to ask you a question. How many of those people on earth can code? And it's something like 50 million or 20 million, 20 million to 50 million. What's really fascinating about what's happened with these open AI models is for the first time, we are not speaking to the, the, the technology in its language.

It's speaking to us in our language. And he said that, and I thought, wow, this is really what's so valuable at generative AI is I, all my data scientists friends are like, this is nothing new. I've been doing this for years. And I'm like, yeah, but you can write Python scripts. I can't. And so to really democratize it has been so incredible. So Sharon, Nick, I really appreciate you on the Sunday evening. I know you two have an event spirit to get to.

So I'm going to let you get to that, but I thank you for taking time to come in, get in this podcast. I'm looking forward to the show. Learnings GTC was an amazing AI related show. I'm sure Dell technologies is world is going to be as equally engaging. com. com. You want to DM me and challenge for you. Like Keith, you didn't ask Nick or Sharon about X, Y, or Z. com. Talk to you next CTO advisor podcast.