Grok 4, Quantum Threats, and NVIDIA's China Strategy | Infrastructure Matters
Transcript
Hey everyone, welcome to Infrastructure Matters. We are back and I know we've been gone a bit here. Um, there has been a mass of travel and also some technical kind of issues. So, we've had a few glitches there and not recording, but you know, folks are still traveling around. I'm I'm in Boulder, Colorado. We've got um Keith is there in Tennessee. He's got um he might be frozen here. He's got Starlink on top of his um his his his vehicle there where he's at building another house, right, Keith?
>> Yeah, we're uh we we are talking about infrastructure matters. We just got a well dug yesterday. Uh I'll post pictures of that online. Uh we got a power pole delivered. We're inching our way to being a civilization here. >> Right. And Diane, you are someplace over in Europe, but you're also running on Starlink as well, right? >> That's right. Starlink Mini. Uh I'm in Norway this week, so I'm way way up north. >> Lots of sunlight out there.
Lots of sunlight. Okay. So, we've got things to unpack here. Um I think the big first big one that we want to talk about is Gro 4. , But Diane, why don't you take us through what we got here? >> Uh yeah, so with much fanfare um and with a lot of uh praise from many of the benchmarking firms, Grock 4 was uh uh is quasi released. I mean, it's announced. I don't know if how many people actually have their hands on it.
Uh but there's no question that is pushing the benchmarks. Uh it is a good 5 to 10% higher than than the the next leading model, which right now is is one of the open AI models. uh and they always change but no one's really expecting uh Grock to pull so so dramatically ahead and it's doing some very kind what they call first principles type reasoning where it doesn't try and just rely on all the things it learned it tries to actually build a logical case from the ground up uh so this is kind of a new type of new approach to AI which you know has led to some controversial uh results already um and so we'll see we'll have to see what what sort of um new you know system prompt updates are being made to it But I'd be curious to hear what you guys have have heard about this.
But I I would say it it's it's a big shot across the bow of the major major model makers. >> So what they were showing um especially on the demo there was Gro 4 and this Gro 4 heavy and yes they are both available now. Grock 4 is $30 a month. Grock 4 heavy is $300 a month. So what is Gro 4 heavy doing? um it is doing it's what what they release is that they've got a bunch of agents running at the same time to solve the problem and then at the very then the agents come back together share their notes and figure out what is the right answer to that and that's why they're calling it and that's using a boatload of machine time.
So they were showing how much more machine time the Gro heavy 4 heavy is taking because you're you're kicking off a whole bunch of these agents guys guys going after >> and that that's been the criticism that they're just throwing GPUs at it as opposed to to real model improvements right so that's interesting claim it's not supposed Musk says it's not true but you know we'll see >> yeah that yeah I don't know you know I didn't hear I didn't look at the ne the other side of the the discussion pretty much what I did is listen to the the guys at X um and Grock um talking about what they had done and the demonstrations that they had done.
I think the other thing that was interesting about the um session that they held um that the demonstration is with Musk talking about where we're going and where we're at and where he expects us to be in 2025 2026 and there was two things he's expecting that the new Gro heavy or the next generation of this will start discovering new technologies and that they will finding new technologies and possibly new physics by the end of 2026. six. Um then I think given what we the noise that we saw with Grock this last week um I think his next comment was really super important that we had is that is while he's looking at Grock to be able to formulate hypotheses and determine if it's true or not in 20 by 2026.
The other thing that he's looking at is most important is to maximize truth seeking. Um and but I still don't know if that's going to solve some of the problems that they just had this last week with the Nazi like language that came out which was um pretty significant. So there's a lot of advancements that they're doing but the question is you know can we trust what it's doing? >> Yeah. And I'm looking forward to testing it out. I might subscribe for a month for the Gro Heavy.
I've been using Grock 3 to do what they're labeling as deep research kind of. Can this thing replace the three of us? And the short answer is no, not really. The uh the research links like the the reasoning quote unquote and the uh the opinions are pretty solid. However, every reference link, every source that it gives is a broken source. Is a broken link and to the point where I've had to review some of my own research and it changes the links to broken links.
So, I don't know if that's just a cosmetic issue or systemic. The the real challenge is is that these models use a system prompt. Um, and they're so malleable. Like you can you can tell an AI to to you know dial up your humor to 100 or or you know be more aggressive at searching for you know scientific materials before it and it will follow it. But there's a huge system prompt that that precedes everything you type uh that goes into the model.
And like that that the problem with Grock was because of one apparently a oneline change to the to the system prompt >> and that shows you you know would dramatic impact before you even get a chance to for to get access to the model all this stuff all this tuning has been fed to it. >> So you know how this really affects our models. We're learning that this really really makes a big difference and leaking you know the system prompt is now become like a big you know legal issue that you know and and resulting in battles between these models.
So I think Grock um is is one of the more impressive models and it's going to be very ambitious. Uh you know uh what what I want to you know what I'm concerned about is just the the company and the the the the leadership behind it. Is it stable enough for companies for enterprises to to really bet on? That's my biggest concern right now. >> Yeah. And that that leads me to the other piece that I was looking at this week is, you know, the ongoing reports that are coming out on this talking about the efficacy of AI deployments, how well they're doing, where they're doing, and there's a big mix of them.
And we saw some um, you know, Gartner coming out and talking about, you know, what was making it, what was not making it. But there was a another report that was recently done by Harris, the Harris pled with 500 respondents and they were saying that and this was for another company granted but that their um percentage of have fallen quite a bit um and to the point that we're doing you know maybe it's down to 50% or less that are being deployed. maybe only a quarter of the initiatives are actually making it out there which is very very you know there's not a whole lot of them that are making it out and part of that I think is because of all the reasoning um needing to have the somebody you making sure the data is right that before you deploy it because otherwise it can really really affect the brand significantly.
Um and then what one of the other things that came back is that they're talking about you know where is the deficiencies right now and it's the obvious deficiencies are the data pieces of it and then the people pieces of it and making sure the it's tied to the critical decisions that that are are are good enough um are meeting the needs of the company. So yeah, that is so we're still, you know, we're still cautious about what's happening and what's coming out. Um, and that eventually, you know, we're going to get there.
Um, all right. >> So the real concern, what I'm hearing right now is uh there's been a lot of disillusionment around agents that uh they they're building these tech stacks, but the amount of actual agent behavior is very low. And this is what we're seeing in our own research right now is is the tech stack just got completed for you know the initial tech stacks for agents. Very little business use cases are have been implemented with agents yet. Um and you know it's like it's like creating a jury case that's great that the agent can do that for you but that's not a major leap in productivity.
What you really need is like the the the case to be resolved and researched by the agent as well you know. So that's interesting. Well, that's speaking of infrastructure. Um, you know, quantum's been always off in the distance, five or 10 years away before we really have to worry about it. But those days are getting shorter and shorter. And that we're getting not just quantum uh chips now. So, instead of having these big, you know, super chilled rooms uh with a few cubits inside them, we're now getting cubits on chips.
And now they figured out how to chain the chips together. We're getting real actual not toy quantum computers now. And that means that cryptography is under threat and that is the basis of cyber security why we can encrypt our credit cards and uh why we can digitally sign things and why cryptocurrency is safe and on and on and on and with the in a postquantum world uh those things are not safe they can all be viewed in fact we're vendors like HPE have reported to us that they see people collecting data in anticipation now of being able to decrypt it soon um that is that's scary and so uh for the very first time We see postquantum uh cryptography has risen.
Now IT Pro just announced that 77% of companies either have it uh in pilot or they're trying to figure out how uh to start piloting postquantum uh cryptography uh cyber security solutions so that they're not suddenly standing naked out there in the digital world because that's what's going to happen. Uh so that's a that's a big shift and the first toolkits now the postquantum security evaluation toolkits have now arrived from many of the vendors and it's an interesting time folks. So that's a piece of infrastructure has to be revisited now.
Everything everything you have has to be re revisited. Uh what what else do we have? Um I think up next um is uh Nvidia. So this is a interesting piece of news because I had some people challenging me and they bring up some really good points. So Nvidia is going to offer a chip that is um that is friendly for US export restrictions. Um not as not as capable but within you know within the ballpark of of of of their um the GPUs that they're selling in the international market.
Um they're offering something that's fully CUDA compliant and CUDA has a lock on the market even in China, right? And so they haven't been able to ship chips there because of US export restrictions. they're going to make a China friendly chip now. Um, that's great. But what really re really is interesting to me is that Nvidia is a Taiwan company and they're now selling going to sell, you know, the most capable ship. They're allowed to sell, you know, GPUs to China, which technically is many people are concerned that they're going to invade Taiwan or at least blockade them sometime in the next two to five years.
um you know, they're constantly doing drills and surrounding the island. Um yet now Nvidia is going to sell them chips. So with that's not maybe not necessarily the smartest thing to do, although maybe it is just to take their money. I don't know. Um the uh but this this is fairly big news and uh and I call this the chip that will make Nvidia a $5 trillion company. Um, I think they were going to sell a lot of them because China has been constrained in AI because they haven't had CUDA chips which most of their AI software still uses.
>> Well, we'll see because he didn't Jensen meet with uh Trump yesterday in the White House. >> Yes. I don't know. I don't know the outcome of that one. >> I haven't heard any noise about that. Yeah. >> Yeah. Trump Trump has been um he's been very uh triumphantly announcing Nvidia's, you know, first first $4 trillion company. That's why I was joking that the ship will make him the $5 trillion company because they're going to sell to one of the biggest markets in the world.
You know, millions of these chips for sure. >> Um Trump's very proud of Nvidia's uh you know, they're they're on a US stock their US stock listing is has the largest market cap of any company in history, right? So, it is it is worth uh you know, it is triumphant type of thing. Yep. So a couple things that um I was noting recently is we are continuing on this march through the as the market market New York Stock Exchange etc just continues to take off you know with Nvidia but it's not just Nvidia it's a whole lot of other companies that are rolling um TSMC um Taiwan Semiconductor their sales were up 39% but you know that's indicative of what everybody else is doing because you know they're the one that are manufacturing a lot of things like Nvidia and Apple and and others.
Um, another one that was noted was ARM. Um, basically saying they're they've had a 14-fold increase in um, chips that are being adopted and that is getting driven by hyperscaler adoption of ARM for um, general purpose and AI workloads. So those are those are picking up um and uh, you know, it's the situation that it feels like you know this AI stuff floats all boats, right? um wherever we're going um super micro um talking about opening up more um manufacturing capability over in Europe um and to meet the demands that are over there um and that you know they this while their Q1 growth was not as much as they wanted um the stock exchange folks wanted it to be um they're still marching along and and growing significantly and and I think then the last thing I'll wrap it up with here is that IBM announced um the Power 11 um which is both server chip you know and um the power 11 is will be their IBM power with next generation high-end mid-range entry-level servers but that chip is also going to be um the power 11 will also be um support the IBM spire accelerator um which is IBM systems on a chip um and that is an AI intensive inference for AI intensive inference workloads and that's going to be on you know will be in power 11 Z as well as well as um some other areas.
But the power 11 is well used by health care and the financial community. Um it's known um it's for super high availability. Um it's in the I don't know 8 ns or some sort of ridiculous number like that. Um that it has the you know availability of as well as the cyber security capability which gets back to you know what you were talking about what's going on with the next generation of cyber elements that we have to be focusing on. So really seeing that I don't know I I just don't see an end to this market going um I mean >> well I mean it should be I mean this is a good you bring up a good point which is um you know the AI is probably the biggest thing that uh will hit during our you know since since the internet maybe and may and some people believe it'll be bigger than that everything can be automated all human labor can be you know improved or you know um supercharge and you're not limited by the number of humans you have, right?
So the economy can technically grow. This is why the industrial revolution was such a big deal. Our economy was listed limited by human and animal muscle power. And when you eliminated that as the constraint, you can build as many machines as you want. That created an almost unlimited economy back then. AI now says your economy is not limited by the amount of people, knowledge workers you have because AI can now do the same thing. we can create unlimited number of of you know AI digital labor bots you know >> um so potentially it's the biggest thing ever so yeah we should see an incredible boom in the market and you know that's not going to be good for inflation um and we are seeing signs of it I mean there companies that have been disciplined there are companies are reporting results with AI that are good um there are some that are reporting minimal results in our own CIO survey we're seeing that it's a small in our also recent CEO survey on on on on AI.
Uh we are seeing that that you know the the bold leaders that are not over micromanaging are actually reporting that they are seeing results. We got hard numbers back from those folks. So yeah, I mean I think it's going to happen. It just you know like all all of this is going to take a little bit longer than most people think, but it's going to be bigger than most people think. >> Well, and Microsoft did talk about their savings this week.
um 500 million of more saved in their call center because of what they're doing with AI. Um and um you know on the other hand we also saw some reporting saying that the call center is not working. So you have you know there's one is saying hey it's working really well. I haven't called their call center at Microsoft so we'll see how that does. Um and others are saying ah this is not working. I mean it's it's giving the wrong answers or it's frustrating people or all those kind of things.
So you know it's uh >> but those happens with humans too right. So the question is can they get it to be better than humans and I suspect they can right you know over time. >> Absolutely. Absolutely. So I don't have the topics in front of us because that got wiped out when I had to pop off. So Diane, you're you're back on um in terms of lead talking. >> We're good at we're good at rolling with the punches here.
>> Yeah. Well, I mean, I can build on top of what you were talking about with semiconductors and DSMC and ARM. And it's a little sad to see because I remember I, you know, unfortunately I was around when the very very first chips came out. I'm so I'm dating myself, but I remember the 80008, right? That was that was the first chip and then, you know, the 8080 that was the one after that that was such a big deal. And Intel has been the bellweather, you know, virtually helped create this industry.
Uh and we just heard the news uh this week that they they've dropped out of the top 10 of chip makers uh after another big round of layoffs and more shrinking market cap. Um and and they're no longer the chips that drive a lot of the best computers in the world. My my MacBook in front of me here is running, you know, an M4 processor. um you know that's that's that's not Intel technology because they just couldn't they couldn't produce uh the high performance enough chip and they couldn't do it at a even if they did they couldn't do it at a price point that would be acceptable and vertically integrated companies are seem to be part of the future everyone's building custom silicon now and that's not been good for Intel and now the world's shifted towards GPUs um and you know Intel does not have an equivalent of the CUDA stack um I mean they have it they have AI chips that developers are not using it and that's the death now.
So when developers are not writing code for your chips, that's a really serious issue and that's that's that's the kind of miscalculation. They didn't know that this was going to get so big. Nvidia bet was the chip maker that bet on where the market was going to go. Uh they didn't know that this was going to happen in my opinion. They did, you know, they they were a gaming company until quite recently. Then they and now they're a data center and AI company is where most of the revenue comes from.
Um but it was fortuitous that they built they they chose the the right model and and created the right stack that the whole world has beat a path to and and and and tell missed that turn in the road. They you know >> well and I think there's two parts to that is not just the the GPU situation but we're seeing you know AMD coming out with really solid technology that has given for the first time Intel you know real true competition into the core business that they're in.
So they're not able to maintain their market position in terms of owning all of the you know traditional servers that are out there. There are other options that are there and clients are opting for those areas and um those technologies that are out there and it kind of opens opens the door to some new markets. I mean it's to me it's amazing how much changes. We think things are really established. We think vendors are very established and own the market. And yet here we are, you know, in in a very short period of time seeing Intel fall from um the absolute leader to uh off of the top seven.
So very very sad, but um it's and it's not that they can't come back. I mean, it's kind of like IBM, right? IBM was was dead more or less. Um, even in the last, you know, executive they have before, you know, the current current executive staff. >> Oh, yeah. I had I had read them off and read them off. You >> can do it. Intel can do it, right? Yeah. I hope so. >> So, so that's, you know, our cheer cheer for Intel to get back there and and get back up on top.
So, >> yeah, exactly. Uh, well, we're we're we're getting towards the end of the show here. Uh, a lot of interesting stuff happened and um I'm not sure I think Keith's out on his roof trying to raise up the height of his Starlink so I'm not sure he's going to join back his uh join back. So I don't know if you can talk a little bit about the core acquisition um Kimberly or and you also also super super micro had some news this week.
I did not see the core acquisition that that that was an investment that was being done right by um I Oracle dropped a significant amount of money >> my coverage area so I I was hoping maybe you could talk a little bit about but I can I can uh do a core we've acquisition of course scientific looks like >> so that's all I know >> so so I think we will wrap without Keith's Keith's commentary we'll catch him next week and Um, we'll see what kind of technical issues we'll bump bump into that time.
>> Yeah. All right. Well, excellent. Um, well, we'll um I have uh my schedule for the next few Fridays isn't gonna be around. So, Kimberly, I'll see you next week. Um, and >> thanks so much. >> Thank you. >> Thank you, everyone. Bye. Bye.
