Episode 703: Your AI Capacity Plan Depends on More Than GPUs

Machine-generated from the episode audio and not hand-corrected, so names and technical terms may be imperfect. The audio is authoritative.

A pretty good habit of getting these in. We're at episode 703 of the CTO Advisor podcast. If you're new to us, I'm Keith Townsend, the CTO Advisor. And I'm Calvin Hendryx-Parker, CTO and co-founder of Six Feet Up. So Calvin, we both have a busy day today, so we're going to jump right in. Is the AI bubble finally bursting? And are CTOs going to be left holding the bag for all of these big AI data center build outs? Yeah, this was, I believe, all over the news last week.

Various banks across the industry are basically signaling there's some significant risk out there. I think the big story, the one that kind of caught my attention, and maybe it's just because I'm not a big fan of this organization, is that Oracle is in trouble. They've got, and they're in the negative on free cash flow, which is not someplace companies typically want to be. They've got leases. So when you take leases, they do all this kind of shell game stuff up the thing.

They've got $288 billion in leases that are currently basically not showing on the balance sheet, and half of that, half of its $630 billion contracted future revenue is coming from OpenAI. And so they are building the Stargate campus, you know, in New Mexico, that they told on September 24th, their landlord, that they may have to delay full rent by up to three years. Landlords don't like that. Definitely not. Now, so for OpenAI to even pay back, I'll pay on this, to even get that, say, a 12% return, which is what most of these people want of their $250 billion in invested capital, it means $30 billion a year in profit, not revenue, profit, $30 billion.

And that's not even the worst of them. Well, let's talk about Oracle for a second, because I think it's obvious to say that we both have friends at Oracle, whether they're in OCI or in the traditional business. And I read on a rant a few weeks ago about this. Every company decides what to do with their free cash and what to do with investors' money to expand the business into new areas. I personally take offense to what Oracle has done with their, not just their free cash, but resources from their database and ERP business to cut back those resources in the revenue.

And thus, I believe the promise to those customers on support to fund a build out that these customers are highly unlikely to be customers of this new service. There's one thing to say, I'm making a bet with the contract that you've given me to give you better service. But, you know, they're funding this to basically pay for their contract with OpenAI. That's my rant. Yeah. But there's a foundational issue behind this. That Oracle Stargate campus does not have enough power to generation available to power it.

It is relying on a natural gas pipeline that is coming through Texas, hitting New Mexico. It's been mostly across federal land, which had no problem getting access to. There is a literal half mile stretch that is state owned, and New Mexico is telling them no. That is a problem. That's a real problem. So that's that's that's one problem. Let's talk about, you know, let's give another example. You have a couple on our, you know, little rundown. What's another data center outlier?

Well, we've got the the other big one is CoreWeave. But that's not on my list here. But actually, the there's a bigger problem. CoreWeave. There's a big problem with Microsoft cancelling Nscale's UK site because they can't get enough power as well. So power is a thing. Power is a big thing. And just from a pure data center perspective, people may remember that I ran the CTO advisor hybrid infrastructure inside both Equinix data center and a QTS data center.

As a public service. Yeah, as a public service. And you even then this is I contracted, I think, in 2020. Even in 2020, power was the big driver. If you've contracted with colos prior to 2020, maybe the early 2010s or earlier, the driving factor was floor space. We had plenty of power. Floor space was the big knob to move on contracts. Now everything is dictated by power. I had a full cage at both locations just as long as I contracted enough power.

Yeah, they'll give you the space. They'll give you the space. Now, you know, for a 1U rack, a 42U rack or 48U rack now will hold basically four AI servers with fully populated before you run out of power. I was just at OCP, the Open Compute Project Summit, last week, and they were talking about how do they cater the non-for-profit, how do they cater to the 100 gigahertz, not gigahertz, the 100 kilowatt or lower rack, not data center, but rack. So power is not just impacting these big conglomerates and corporations, but also just from a practical sense, how do I have enough power to run my vSphere server or cluster when I'm competing against AI startups and people taking up all of the power with four AI servers.

Right. There's going to be consolidation. There's going to be people shutting down. But as CTOs, you're probably trying to write multi-year contracts with these organizations to ensure to lock in your compute and power space. What happens when those people now are owned by somebody else? And I think that's a big consideration. Or worse, what happens when they no longer exist? Right. That's the worst case. Yeah, when we were going into the shutdown a few years ago, my advice to CTOs then was to lock in capacity because there was going to be a rush on, I predicted that there was going to be a rush on cloud capacity and that there would be a shortage of hardware.

Well, we're in a much, if you ask me, we're in a much worse situation today than we were during the shutdown period because not only is there not enough hardware, there's not enough power. I was talking to one major networking provider and they were telling me that new, I'm sorry, they were telling me new deliveries are 60 weeks out. So if new deliveries are 60 weeks out, we are now looking at the CoreWeaves of the world to say, hey, I need capacity today.

CoreWeave has taken on a bunch of debt to get GPUs. They seem to be the only company with GPUs. Therefore, I'll default to CoreWeave. Well, they're $35 billion in debt and the insurance on that is a lot of money. Is about 11% per year. That's not sustainable. That is unsustainable. Even with, you know, they're primarily backed by Nvidia. Yeah. And Nvidia pockets are big, but they have their own business and concerns to worry about themselves. Right.

So I guess the question, Calvin, you know, you let's say that you were working in a project and, you know, you unlocked some incredible workflow via software and you're, you know, you're kind of abstracted from the GPU itself. You're developing software and you need these, whether they're on-prem or cloud resources, you depend on these on-prem or cloud resources. Why should you care about any of this? I mean, you're still going to need GPUs for inference and those just are, well, all spoken for, for the next 24 months.

You will not be getting those resources unless you go to one of these providers. It's probably going to need careful, more careful consideration in your contracting with these groups as to what the contingencies are when this happens. And I'm not quite sure on this. Maybe this is a question we'll save for another day, other than supplier diversity. And I think we're in a market for, if not availability, supplier diversity. Unlike early cloud, the protocols are kind of similar, right? If you develop against the OpenAI API, you can kind of go anywhere.

Yeah. You can move to others. I mean, it's about costs at that point, because when you switch over to, say, Bedrock, you're going to be paying more than if you were self-hosting that on a CoreWeave GPU someplace. Exactly. I've been doing labs around this. Forget the lab number, but the basic premise is you need to contract with these companies today. Because I opened up a support ticket with both CoreWeave, Google. Well, not both, all three. CoreWeave, Google, and AWS to request access to GPUs.

And I understand, you know, I'm a couple of person shop, but I never got a response. Not a note. I'm not surprised. I never got a response. So these relationships take time to develop. I talked to one cloud provider under NDA, and they told me that they could sell all of their AI capacity, accelerator capacity, to either OpenAI or Anthropic and not have anything left over for the enterprise. So we're all fighting for what's left over from Anthropic and OpenAI.

Well, and there's side concerns, too, is that there are people who are subscribing to some of these services and trying to use them to distill the frontier models. So the cloud providers have to now put in protections, which is why you most likely did not get a response to those support requests. They are trying to battle these malicious folks out there, the various other labs who are building other kinds of models, because they're trying to just extract that intelligence straight out of the models at the inference layer, and you don't have enough reputation to get enough pull to show that you're not one of these people.

Yeah, and I think one of the practical things from the labs that I have been running, CoreWeave presented last week at AI Field Day 9, which I participated in, and I also presented the 4 plus 1 AI infrastructure model there, which is in the community session. So check that out. But these companies are trying to expand beyond just being GPU arbitrage. And I think one of the things to look out for is that the services are pretty good. I didn't realize this, but CoreWeave had acquired Weights & Biases, which, if you're in the ML community, is a very well-respected outfit.

And they made all these promises about being able to do your training set and your workflow, basically your training pipeline on their new combined service called Forge. And I have to say, I tried it out, and it's pretty sticky. And these sticky services, when you're thinking that you're using the CoreWeaves of the world, just for GPU outsourcing, be careful in your architecture that you're not adopting these services that kind of lock your workflow to these providers. And you can't easily recreate them, whether they're on the public cloud or internally or on another service provider.

Yeah, it's no longer hybrid cloud. It's hybrid workflows, hybrid development with the contingencies in your code to handle moving from provider to provider for the inference. So, Calvin, you have to run to a client meeting, but I wanted to give a chance. You haven't really told us what you're working on. Like, what's interesting over at Six Feet Up, other than me seeing Chris Williams all over the place? I think Six Feet Up is really just a cover for Chris Williams.

For Chris to just fly everywhere and hang out with community. Oh, no, and he's doing a lot of that. So for those who don't know, Chris Williams joined Six Feet Up team recently, and he's been our field CTO, but he's been focused a lot on some partnerships. So if you're in space and you're looking for someone to talk to about software projects, Chris should be a really good person to connect with from a partnership standpoint, because he understands the enterprise space really, really well.

I'm so excited to have him on the team. And I just got back from Malibu, California for the XPRIZE Visioneering event, which actually was really fantastic. org. You may have heard of the Commercial Spaceflight XPRIZE, which was the very first one. But the more relevant, most recent one that got awarded was the Wildfires XPRIZE. And that's very relevant today because there was a wildfire out there while we were in California. And so they're looking at building challenges and awards for people to go after some of the world's hardest problems and that are going to make a significant impact on humanity.

And that aligns with what we're doing at Six Feet Up. We're going after some of the hardest problems we can find to go solve. So that's been a lot of fun recently. So we're almost in Techvember. I'm going to call it Enterprise Techvember, where there's a series of events. I'll be at Dell Analyst Summit, KubeCon the week after, and then Supercomputing is happening here in Chicago. Then at the end of the month is AWS re:Invent. I'm sure you'll be able to meet Chris Williams or even Calvin, especially since he is an AWS hero at re:Invent.

And you'll be able to meet me at any one of those events if you're there. And I think we're all at KubeCon too. We're all at KubeCon. Maybe there's an opportunity for Chris to join in and join in the podcast. I've only had him on four or five times. All right. com. And Calvin, Six Feet Up's URL? com, all spelled out. S-I-X-F-E-E-T-U-P. And check us out.