Servers Will Never be the Same!
Transcript
>> Welcome to VMware Explore 2022 with the CTO Advisor studio. Come on in and consume some content. >> I'm Stephen Foskett publisher of Gestalt IT. There's a new technology coming called CXL and it promises to absolutely revolutionize server architecture. But first it's going to have a really unusual impact on a few very specific applications. By 2025 servers are not going to look anything like they look today. This is the Gestalt IT Checksum. >> This is also the CTO Advisor's CTO studio from VMware Explorer, 2022.
We're live at the end of day two. Steven, it's been an amazing show. I really appreciate you coming on and kind of breaking down CXL to me because it sounds amazing. But first I think we need to get to what is it? >> Well, I think that's the interesting thing, Keith because it's actually kind of not all that exciting sounding. So, you know, PCI Express, right? >> Right. I mean, that's already around and a lot of people are using it.
Basically it's the bus that's inside your computer. Well, you also might have a Mac and you might have experienced or PC with Thunderbolt. Right? Well, Thunderbolt is pretty cool because it's fast and it's low latency, but the secret of Thunderbolt how it's fast and low latency is that it's actually PCI Express over an external connection. CXL is kind of like that. Essentially the idea is, we want to hang different devices off of a PCI Express bus so that they can be accessed by the host as if they were local devices.
Now, sometimes they're going to be local devices like NVMe storage or this new generation of CXL based memory expansion devices. So we've got Samsung and SK Hynix and others are already introducing CXL kind of version one memory expansion devices, but in the future what happens is those devices don't necessarily need to be inside the server anymore. So imagine if your NVMe drives or your memory expansion or your IO controllers were not in the server maybe they're in another shelf in the rack. Now imagine that the CPU and the memory and the IO and the storage and all those things they don't even need to be in the same thing anymore.
What is a server anymore? Essentially you can explode the server and build whatever you need. You can compose it based on the resources that are all directly attached over a CXL PCI Express fabric. This really revolutionizes, changes the entire shape of servers. And that's really what I'm excited about long term. >> So let's talk about that long term. Long term, what you just described sounds awful. Like awful, like HPE's the machine. >> Indeed it is. Honestly, this is very like HPE is the machine.
Essentially what we're talking about here is doing what they wanted to do, which is basically make a composable server. The problem was that it was a very limited set of hardware and software that they were talking about. The brilliance of CXL is that Intel created this protocol essentially that would enable this stuff and made it into an industry standard in the kind of way that only somebody like Intel really can. And every company in the industry has signed on and no joke and all of the competing standards except maybe NVLink have been subsumed into CXL.
So essentially, they have created this standard for something going forward that very much could be the equivalent of a Bluetooth or a USB or an x86, except for disaggregated servers. >> So obviously, if anyone has done any type of disaggregation where we're putting IO external to the CPU or the same chassis that the CPU is running on we run into this problem of latency and transport. Where are we at with not just the protocols around CXL? So PCI-e Express is great but I got to defeat the problem of latency and distance for this to be a rack of miscellaneous components.
>> Yeah, absolutely. There's a bunch of technical problems with this and technical reasons that it hasn't been done before. So one of the things about CXL is that it could be used, I mean, frankly, over PCI Express 3 but it really isn't. It's really not going to be real until we get to PCI-e 5 or even PCI-e 6. And the reason for that is essentially we need ultra high performance ultra low latency in order to make it make sense to have memory live on the PCI bus instead of on a specialized memory bus same with storage, same with anything that's external to the rack.
The other reason that we need that is because we need to have the PCI Express not just be a point to point link, but be a switched link like a network or a fabric. And in order to do that, we need a whole new generation of PCI Express chips that will enable that. And that's not really coming until PCI-e 6. So there is some time still before this takes over. >> So I think sometimes we forget when we think about PCI-e bus, it's a bus and not a switch, not there.
There's a reason why it's a bus and not a switch versus 'cause we don't have to worry about collisions all this stuff that we have to worry about in ethernet and share connections. So I'm pretty excited about it as well. I've been doing a lot of content with Micron about just the data center of the future painting this out and handing towards CXL being the future of this is the deepest dive that I've gone into it. I hear you, that vendors are coming onto the standard but are there going to really, I mean are we going to see products not just from the memory expanders of the world?
Are we going to see Dale, HPE, Lenovo am I going to see this in a ProLiant? >> Well, that's really the key question here. And I would admit that this very much could go the way of I don't know, Thunderbolt and be sort of a niche product for certain customers. One of the customers that's really pushing it is the hyperscalers. So they have a fundamental problem. Somebody like a Meta has a problem in that they want to make sure that they have all the memory channels populated in order to get the most performance out of their CPUs.
But the problem is that memory chips only come in certain sizes and memory chips that are big are really expensive. So if they're going to fill all four channels with 32 gigs of memory or something, that's one thing. But if they're going to put like 128 gigs on each channel well that gets to be a pretty big portion of the server cost. What they're finding right now is they actually have to equip their systems with way too much memory in order to equip all the channels.
They can use CXL as a way to right-size memory and save a lot of money. And so they're very excited. So what I'm trying to say is that hyperscalers are very excited about memory expansion using CXL but that's not transformative. That's just a bandaid on the problem. What we need in order for this thing to take off is this sort of idea that you have this fabric where there's all sorts of devices talking to each other. Composability another thing that we have to worry about is sort of ownership.
So if you've got two CPUs that both want to talk to the same peripheral how do we deal with that over this fabric? Just like an ethernet and there's all these things that have to be dealt with before it becomes real. >> Yeah. I was talking to Rebecca Weekly from CloudFlare about this very topic and the concept that GPUs typically don't have the same security mechanisms and that CPUs when it comes to memory because typically GPU's memory is just a GPU is just has to worry about GPU memory.
But what happens from not just application performance perspective but also a security performance security issue. When I can now get a GPU on the same memory bus as my CPU memory, and I can give a GPU a terabyte of RAM from a AI ML perspective, super exciting, but from a security perspective, seems like a nightmare. >> Yeah. Well, absolutely. And the nightmare comes when you have basically inconsistency between caches, between memory. In other words, what if this CPU thinks that this region of RAM has this value but this CPU thinks it has some other value because he think he wrote over it.
And then this guy doesn't know that and they're both accessing the same memory or the same GPU or the same IO card and expecting the same thing. There's no co coherency there. That's the thing that CXL is really all about is about cash coherency. It's about making sure that devices all will see the same data in the same place. And that if I write there that I'm going to tell you that I wrote there, and then you're going to know that I wrote something there and you got to go read it.
That's the problem. And that is something that needs to be overcome. And then of course there's the security aspect too because if you're composing servers with the same RAM, for example, and two servers have the different security domains but they're seeing the same memory, there's all sorts of possibility for leaks and that's not good. So of course you have to solve that problem as well, access control. So there's all sorts of things that need to be dealt with. But the good news is that all those things are that are specifically being addressed in the CS CXL spec.
>> Yeah. The thing that we didn't touch on is the mismatching of memory, the demise of Optane and how the industry just kind of shrugged their shoulder, and like, oh well, CXL. >> Yeah. I think that that's an interesting thing because what killed Optane really was the fact that it was shackled to Ice Lake and Ice Lake was late. If it hadn't been for the fact that everything anything that could use Optane was delayed in the market then Optane, we might be in a different situation but you know, you can't cry over spilled milk.
>> I experienced a problem in my data center. I had gen 1, gen 2, and gen 3 Intel scalable processors. And I got a bunch of Optane. We did a project, a shared project together. I'm like, I can use this in two of my systems but I can't use it in one, that really does hamper the effectiveness of the platform. But again, can't cry over spilled milk we're moving forward. >> Yeah, exactly. And that's really the challenge here with CXL as well.
Is that not every server's going to be ready for it. So the next generation Intel and AMD CPUs are going to be ready for sort of the first generation of CXL, the memory expansion, that sort of thing. But it's going to be the next generation with PCI-e 6. That's going to be able to do this whole composable infrastructure if that gets delayed. If that somehow there's some mismatch between PCI switching chips and CPUs and all that kind stuff all this stuff that we're talking about could be derailed just like Optane did.
>> So last question for me is we're at VMware Explore 2022. I've talked to a lot of VMware product developers, product managers, et cetera, no conversation around CXL. >> Nope. I don't want to be weird about it, but nope. >> Nope, yeah. Like I talked to a lot of folks you would think from a ISV perspective, VMware would be one of the main contributors or people in conversation. So maybe we haven't been introduced to the right folks. Maybe there is working groups within VMware.
When I get to go back to talk to the folks at Intel and Micron I'll ask them about involvement from VMware. Maybe it's just a group we just don't have access to yet. >> Or it could be something that they're working on that they just haven't talked about yet. I mean, you know how VMware loves to do their project this and project that. You know, CXL might be project blueberry that's coming in the next version. We just haven't heard about it yet but I can't imagine that they're not at least working on the tiered memory aspect of CXL because that could be really powerful in a vSphere server and the whole idea of disaggregated servers.
Well, I mean, that's everything that vSphere is going to run on in the, I don't know, version 10 or something generation. So I really hope they're working on it. And I hope it just means that we haven't heard about it yet. >> I agree. Well, Steven, I think that closes out for us. We're officially, we're doing the thing that in tech field day and CTO advisor community that we're doing we're guerilla the show floor is closed but they haven't given us the hook, which meant that we could sneak in this extra special content.
com. My DMs are open on Twitter, @CTOAdvisor. >> And as for me, we're going to be publishing this as an episode of the Gestalt IT Checksum which is my editorial that I do. com and you can find me @SFoskett on the Twitters.
