The Impact of Deepseek on Enterprise IT AI Strategy
Transcript
all right it's been a while since we've published a podcast because we're trying to only say stuff when there's something to say and man oh man what a week there is absolutely something to say we're recording this the week after deep seek has dropped and I have no better guest to talk about this couldn't possibly be be a better guest than Diane hen Cliffe who is our uh VP and practice lead for our CIO practice here at the futurum group Dian I think it's the first
time you on the program right um first or second I can't remember yeah something like that well the the the the details of which I don't know matters but I wanted to bring you on because you had a really great informative post on social breaking down deep seek and I wanted to sherff our mainly infrastructure focused audience why deep seek matters etc etc but let's start out there what is deep seek and why does it matter so it is definitely the shot her across the bow
of the of the current AI Darlings you know that open Ai and anthropic and and Google and all of those folks um it is a is a large language model it is from a company in China um it is uh a very high performing model it's the highest perform performing model that's ever come out of China um they've been producing some llms but when you look at the benchmarks um you know there there's now very well- defined benchmarks that allow people to to quantify how good
is this new new language model there's all these benchmarks around reasoning and other things that you can put make them go through and deep seek landed at number three right out of the gate which which never happens do never have it's unprecedented to shoot right to the top of you know push your way past hundreds of other models and say you're as good as open AI across a wide SWA of metrics and number one in things like coding and math and um that if that wasn't
a big enough deal um it reported they took took them 45 times less training effort to create this model so it's a dramatic reduction in cost and time uh which just the big input for these models that's the mo around the whole industry is that you have to come up with hundreds of millions of dollars to train your model and they somehow haven't had to do that and uh if that wasn't enough there's it could run on a regular computer I I mean a regular high-end
you better have a Mac Studio or something uh you can get you know 01 performance that's um open ai's Flagship model on your on a personal computer uh and it costs Pennies on the dollar compared to open Ai and and their ilk um so it's it's very disruptive a cost a trillion dollars in market cap to vanish uh from the tech markets yesterday and video's down 177% if you look at their stock price just the cliff because if they're if they're if they're right if they're
accurate and it's not clear that they are there's a lot of speculation that they have 50,000 h100s in their back pocket that they actually use but they can't say that do the expert controls um but is it is re reset everyone's expectations about what you can do in yeah so there's a lot to unpack there uh it's a Chinese company as you mentioned and the claim is that they uh did this with basically $6 Million worth of infrastructure that claim might be a little bit dubious
yeah exactly even the case let's say let's accept that it's you know 50,000 h100s one of the things that I wanted to help with context is h100 performance versus the latest Nvidia chipset which is I think the b200 uh I think it's fair to say the b200 is what like four times more efficient than the h100 yep and hard bandwith but yeah right yeah so higher bandwidth more more efficient and the implication of that is twofold one that this company has done some amazing things bar
just unquestionably the second question is what does that mean Downstream because I don't think most of us are going to develop models in most cases I think that that that's still going to be you know a fairly high-end request regardless of the barrier of cost there's other barriers such as Talent time Etc and access to data what does this mean for the downstream for the Enterprise so for the Enterprise it might mean that world class AI is now going to be repriced completely differently now the
big risk is and the joke is that they should if they they really should have called it you know chat CCP right um that this is this is run in communist China this is hosted there uh if you're using the the online version of it and it does seem to to store keystrokes and and your all your history and all your output in on China servers um so but you can it's open source supposedly everyone's trying to determine how open source it really is and you
can put it on your own computers so technically if an Enterprise wanted to get access to absolutely leading class output that would cost a fortune through uh through your open AI subscription Enterprise subscription you can now do this for for essentially free in your own data center right um and so um if you trust the source and there's been a lot of questions it uh um it is censored uh for example you know you can't ask it about T and square it says it won't do
it but you can ask it about anything you want about America politics and it will answer so it's tuned very carefully um to CCP sens sensibilities and um while a lot of people have jailbroken that and said and descried that information is in the model is is obviously clearly censored and um so it's very interesting if you're willing to take the geopolitical risk of sourcing leading class AI from China then the price Mo I mean and the price point makes it very attractive you an 01
level model can do remarkable things produce vast amounts of very high quality output research uh develop software for you um at a level that if you've just been using chat GPT you don't understand what what a what a Frontier Model can do um can produce you know ready working applications with almost no defects uh that are relatively sophisticated based on a spec the push of a button and that's what you'd be able to get with with deep seek in your R1 that's the specific model they
released another one by the way I don't know if you got that Keith they have another model they just released a multimodal model um that's really most more focused on analysis of of of of images text and production of im images and text but anyway um it's everyone's worried that this is going to make everyone's Capital plans this year absolutely upside down but it's great for Enterprises if it makes the price of AI 10 times cheaper which is likely to do do so uh r1's uh
has some very amazing optimizations around it only uses 8bit floating point it had uses um uh extreme compression on the keywords uh so that has something like an 87% Savings in in vram um that those are big deals and those are those will get to every model everyone's going to copy their optimizations for sure at least because it's open source they can see how we how they got it onto a massive model onto a small machine everyone's gonna do that it's I think it's it's gonna
be great for AI on the cons buy side is the bottom line yeah I was talking to the chief scientist over at Nvidia this was a I think at GTC last year and he was telling me that they were actually going in the opposite direction going to smaller bits for to get uh to deal with chunking and all of these challenges with it and if you read the paper and I'll link it in the show notes if you want to go deep and read the research
paper from the deep seek folks it's amazing levels of Ingenuity to optimize uh this uh reinforced learning method go back and watch the 100 days of AI to learn what reinforce uh learning is versus supervised learning Etc but they are standing on the shoulder of Giants and I think as a result the Enterprise will be able to stand on the shoulder of deep seek I just saw a analysis someone had three uh m m uh MC m m MC minis uh or Mac Studios daisy chain
together running deep seek at the same performance level level of a chat gdp1 so this gives the Enterprise an idea what this means for you if you're looking at infering Pro projects so Diane let's put our crystal ball on and help customers really digest what this should mean for their Capital expenses if you're looking at building AI Solutions over the next 12 to 18 months how should you begin to re think your capital investment and in big gpus Etc or should you rethink it at all
I think uh there's some obvious things you should not do um first of all one of those is don't make any long-term don't sign any long-term contracts um you know we'll see what what happens at the very least inference has been made dramatically cheaper now the question is has training been made dramatically cheaper and that's going to CH have big implications because you know we're trying to uh we're trying to beat the market and and create a leadership and AI in the United States by you
know launching massive capex projects like project Stargate um you know 500 billion dollars to create models that no one else can do because they can't train them up uh they don't have a you a training environment big enough to go that deep into the data um this could set all that this could set upset the apple card completely if they have somehow created a you know 45x breakthrough that's basically almost two orders of magnitude breakthrough and and and and training efficiency um I don't think that's
going to hold up on the training side on the INF side it's obvious because everyone's able to replicate those results so I would say don't make any long-term plans don't make any big commitments to budget or vendors because I think it's all going to be different um and and I think it's just going to be it's arms race it's just everyone's going to get table stakes and and we'll be able to reach the next level of artificial intelligence something I I call Super intelligence it's not
AGI um but this is going to allow us to run models that have super high capabilities and can do things humans can't do now right now all AI models that exist can do thing can only do things that basically at the very highest PhD human level they can't do anything that humans can't do yet that that threshold is likely to be crossed uh this year and if not for sure next year and these types of efficiencies are going to allow that so you can get these
super powerful um artificial intelligence that can solve problems that humans and Enterprises haven't been able to solve up to this point that is what what Sam Altman's really after and that's what anthropics really after is AIS that can do things that no one else can and then they they're the only ones that own the AIS that that can do that I don't think they're going to be able to retain control like that I think their dreams are not going to come true so don't get locked
into those vendors keep your options open um the AI space is going to continue to have these kind of dramatic breakthroughs this is a chat a GPT level breakthrough the first one we've seen since chat GPT I think uh you know we're waiting for the the next big one is going to be the one that that has an AI that that can is a super intelligence that can do things humans can't and it's GNA probably be down this Ultra hyper efficiency path so let's end with
a plug for some of the work you've been doing on the futurum group side and this is adjacent to this conversation you came out with a big uh CEO data report around just the success of a projects this is in partnership with Kerney what are some of the adjacent learnings and how will people find out more about this um so yeah we just launched a landmark study on what CEOs are doing in uh with AI going into 2025 what their short and long-term plans are what
do they worry about with it uh what are their challenges in in in rolling it out and we discovered that 59% of CEOs are leading AI which is a surprising stat for us um but we're also finding the ones that that that micromanage it are or over involve themselves in it don't see they report poorer results than than the ones that step back after they set the Mandate everyone must take part in in transforming you doing AI transformation and they're part of the organization um and
then stepping back and just making sure they have what they need those CEOs that do that that take that step back and let everyone in the organization innovate um are reporting higher success rates that said we did a lot of in-depth interviews with CEOs uh talking about what they're doing it's is so top of- mind and they're looking they're going after cost savings right now uh better customer service um and they had great very detailed stories about how you can go in now and just ask
a chat bot you know how how many unshipped orders do I have um how many credits do I have left uh do I have you know what problems do I have with my shipments in this part of the world you just go and get get these answers and and customers love it not having to sort through reports and long dashboards they can just ask the question and get the answer it's um and so it's a crawl walk run thing um they're also keep have an eye
on big strategic you know um what are we going to do to reinvent our business they think AI for the most part not all of them if you're in manufacturing they the only thing it's going to affect operations and supply chain but many of them want AI to rethink and create unbeatable new products and services new business models will generate new Revenue they think AI can do it though they think only think they can do it today but they're preparing for that so it's very exciting
if you want to see it you have to request a copy just go to uh future group.com um you'll find a on our website and you can request to download we we'll get a copy to Diane I appreciate you taking out time on your busy schedule you're in demand people are dming you asking you to appear on shows and explain this deep seek disruption to everyone I appreciate you explaining it to the audience links to Dion's socials will be in the show notes if you want
to learn more about the futurum group you can find us on the web futur group.com Diane also published just a couple of months ago he's been a busy man a CIO data set with equally interesting uh insights you want to find out more about the CTO advisor of course we're still publishing our micro Blog the CTO advisor.com talk to you next CT advisor podcast thanks Dian thanks ke
