CTO Advisor 056: Promises of Intel Optane & 3D xpoint

Machine-generated from the episode audio and not hand-corrected, so names and technical terms may be imperfect. The audio is authoritative.

com. You're listening to episode 55 of the CTO Advisor. And we're off the heels of, you know, Intel's Optane drives have been on the market for a few months now. We get to talk to James Myers from Intel about SSDs, data, and a bunch of other stuff. This is a data-driven podcast, you can say. So I'm gonna give it over to Mark because I'm an imitation storage guy. Mark is the real storage guy. So we'll give it over to Mark to introduce James and take it away, Mark.

Well, maybe by the time we're done, you'll be a real storage guy. That is possible. So James, you work at Intel, right? Why don't you give us a few seconds and tell us what you do? Absolutely. So I'm a solution architect here at Intel focused on non-volatile memory solutions. So pretty much means anywhere SSDs go into the mix, which goes right to storage, I get my nose in the middle of it and talk to people about how cool they are and why they're gonna help you.

It's the coolest thing ever. Optane is a brand name for Intel's data center SSD that uses what's called 3D Crosspoint memory. So 3D Crosspoint is a new memory technology, but it also uses then, has a controller and some firmware and some validation around it. But 3D Crosspoint memory is a new non-volatile, high performance memory that has attributes in terms of performance that are much closer to DRAM type speeds, but it has the non-volatile aspects that you get from NAND memory. And it kind of smooshes the two together and it creates a new class, some people refer to as storage class memory.

And when you take this 3D Crosspoint and you put a cool controller behind it and you package it up into an SSD and you have all the goodness and reliability that Intel brings, we've got what we'll call the world's most responsive SSD, which is our Intel Optane, oh boy, long brand name, DCP4800X SSD. Okay, so I think you just confuse the heck out of a few people by talking about storage and memory at the same time. I think that's really odd for a lot of people.

So storage class memory, what does that mean to the industry? Well, it's really this cool new characteristic where you can have speed that's close to what they're used to with traditional memory or DRAM, not exactly the same, but trending, and then persistence that you would normally have for storage. And when you merge those two together, it gives you an opportunity to use it differently. Okay, so is it to the end user? Is it a hard drive or is it memory? Like, how does it present itself to the user?

Are we doing read-writes or are we doing load-store? So the product that we have today, the Optane SSD is exactly that. It's a storage device. It sits on a PCIe interface. It does read and write. But the memory technology itself, and we've talked about it inside of Intel, will very soon be available, I think we said next year, available on a memory bus where you'll do load and store into the technology. All right, so you made this really fast hard drive right now.

How, and you're getting to the point where it's gonna be memory. How's that gonna change things? You know, applications, OSs, they're not really designed around persistent memory, right? They're designed around read-write onto a file system, typically. Absolutely. Applications for things that they store are read and write. They're kind of architected around the arcane spinning rust that is a hard drive in that they're gonna go say, hey, write this and tell me when you're done so I know that it's persistent.

And I might wait around for that or more than likely I'm gonna go do some other things while you go ahead and write it. And when I have to read something, I'm gonna try to get it in ahead of time. I'm gonna batch it into memory so that I know that it's there when I need it. And they're just expecting these very long millisecond level of latencies for the accesses. And some newer classes of applications have really realized that when NAND SSDs hit the market that there's a lot that can be gained on actually utilizing faster storage.

And you've seen applications start to emerge and it was a little bit easier because it really didn't take a lot of software rewrite. But the ultimate goal would be to not have to, as in a database as an example, the ultimate goal would not to have to have a separate write ahead log. It's the way databases work today. They have this log where they record every single transaction so that it's persistent. They record it in memory and that's where their working set is.

And then they record it on this log and they just take a snapshot. It's like a ticker tape on a old cash register. Here's another sale, here's another sale, here's another sale. And the whole database is limited by the performance of persisting each of those transactions in that log. And they do that because they need a persistence model. And every now and then they'll take a picture of the working set, they'll take a snapshot. And then in the case you lose power, you'll normally will take your last snapshot and then you'll take your ticker tape or your log and you can rebuild to where you were and now you're consistent.

You didn't lose any transactions or data. That's the way a database is architected today. But think about it if your memory or your working set was non-volatile, you may not even have to have that log. You could totally change the game. Normally, way back in the day of let's say last week, I had to create a tertiary copy of the database and then create queries against that database, which that slows up the speed of business. I'm not getting real time data analysis off of my most current data.

So that has a limited approach. How do you envision Optane and 3D Crosspoint as a technology solving that business challenge of being able to drive analytics and business decisions off of more real time transactions? That's a really good question. So when you look at that, just to your point, they're traditionally making a copy of the database and when they make that copy of the database, it can be taxing to the active system, to the transactions that are coming in as they make that copy.

And so normally they'll copy it out of storage and into other storage, and then there's a very long delay first to do the copy, then to load that into another system, and then to start doing the analytics. And so you've already seen people trying to get around this by putting in new sequel type streaming sorts of databases where they will take a look at the stream of data as it's going through, and they'll look for certain event markers, and they'll have sort of a first in, first out storage buffer that they'll be storing some of that data.

And then when they see one of these markers, they'll say, hey, 10 minutes before, 10 minutes after, we'll take a snapshot of that, we'll copy it off somewhere else, they can go analyze it, right? And then you're seeing all these new languages come up, and that's a very creative way to do it. But the folks like SAP HANA say, well, if the whole working set was in memory and you could look at it in a columnar form and in a row form at the same time without disrupting anything, that's the panacea.

But, you know, the challenge with even that is that memory is expensive. And if you have a huge amount of memory, you have to back it up to something fast, you have to take those snapshots, you have to be able to work in and out of it, you have to be able to read and write it at the same time. So if you have a storage device or media that can be read and write equivalently, like NAND is not, NAND is slow to write and fast to read, but if you can read and write it equivalently and do it with a very consistent responsiveness, because you're not managing where leveling and all of these other sorts of things, it can really enable you to use a storage media instead of a DRAM.

And so it changes the game of doing these copies and these snapshots and these little pieces of time, and you can really get a broader data set. When you get that bigger data set, you get better answers. So essentially, you're looking to store the entire data set in what would appear to be RAM, but is really like a new tier of memory so that you get faster response time. So you're not really having to worry about necessarily making that copy and querying a secondary copy of the data.

Absolutely, that's definitely the vision. And even just having a bigger memory space to work in is where it's all starting. And so this software can now utilize this device like memory, it's not persistent. You don't get any of the persistence benefits yet, but this is where it's starting, right? Bigger pool of memory, you can put more stuff in cost effectively, and then you have the bigger working sets. You start to get the better intelligence. You're seeing that today. And then over time, the applications will catch up and the interface will catch up to be a memory type interface and the applications will be redesigned and all of that will come much more seamless as it gets introduced.

Then the benefits of the bigger working set plus the benefits of it being non-volatile and this whole new infrastructure. So how seamless is that? How seamless is it today? With Intel Memory Drive, there are certain applications where it's very seamless. I mean, we've got examples with machine learning type workloads, with database caching type workloads. You'll see in the infrastructure a large layer of caching servers that sit in front of a very expensive database server and they'll manage most of the read transactions at a much lower cost than having the reads come off of that database server.

And so in a layer like Memcached or Redis in front of this database. And we see in those examples, those are usually nodes full of DRAM. And with that type of workload, with the Intel Memory Drive and the Optane SSD and some amount of DRAM and a bunch of Optane, you can much more cost effectively build bigger Redis nodes and have more of the database in each of those nodes, which means a higher hit rate of cache, which means then you're supporting more performance and you need fewer of those nodes to support that caching layer in front of that very expensive database on the backend.

And that's just one of the ways that we see this technology is seamless today. As an example, we've done some really good work jointly with VMware. It's been actually, there've been an analyst company called the Evaluator Group that's validated it, where they'll now use it, instead of using it like memory, that's the mode we just talked about, they'll use it like a fast storage device. And they put this SSD, this Optane SSD in a caching layer. And then they'll use NVMe NAND-based SSDs in the data layer.

And they're able to see amazing benefits. So either you get memory-like performance or in this case in storage, you see two and a half times the number of VMs that you can support per server, assuming that your storage or CPU limited and not memory limited. So to circle back to your point about the BIOS, to support something like this, you're gonna need broad support from all the OEMs, right? To actually make the BIOS support it and things like that. Luckily, you're Intel, right?

So you're in a unique position to do that. Actually not, it's like a hypervisor. So as long as we do our compatibility homework, they don't have to do anything special to present the drive as Intel memory drive with the Intel memory drive software. Just like the same thing, they don't have to do anything special to make VMware work or KVM work. As long as they're abiding by the same laws and rules and they're planning to pass things off to the operating systems the same way they always have, that our software just works.

So let's go down this rabbit hole a little bit because we've abstracted away in a sense the physical BIOS and et cetera. We're kind of saying, okay, just as long as you follow standards, this should make things better. e. ARM. Can we take this drive technology and this memory technology and apply it to an ARM system? Oh my goodness, you're asking an Intel guy about playing in an ARM environment? Holy sacrilege. Come on now, you know my answer to that.

This is an Intel world. That's what we're all about. No, I mean, it's an emerging technology. If it's a storage interface and it's a standard storage device, there's no reason that an Optane SSD won't work on any other platform that has an NVMe storage interface. The Intel memory drive technology software is specific to the environments and applications and operating systems and BIOS. So I'm not aware of any ARM-based servers that follow those same rules, but maybe I can be educated there.

Awesome, so today, it's a hard drive. Future, it's memory or a hard drive. How far away are we from that future? Well, you know, we've announced before that we expect what we're deeming right now, the Intel DIMMs. We haven't given it an official name yet. We came out at SAP Sapphire and kind of showed a proof of life with SAP where they were running their HANA database on this device. And we said it's coming next year. So I'm gonna ask a really tough question.

We didn't prep you for this. And this is, so this is truly an analyst and user type question. The folks at Wikibon, which is a popular analyst firm out in the infrastructure space, kind of came out and said, you know what? The Optane as a drive technology, since it's not gonna be consumer facing, can't really get the scale to benefit or survive. There's not enough performance difference to justify it. What do you guys think about the performance difference between the Optane drive and the traditional LAN based drives?

Is there enough performance delta and to justify kind of this as an ongoing business? I know as from Intel, this is your job. So obviously the answer is yes, but can you give us a little detail around Intel's view of where you play in the market? Well, I will tell you that there's no question that Intel is invested in this technology and it's not just me and my job. You know, Intel is really focused on the concept of the data center and the data, you know, the world revolving around data.

And getting data closer to the CPU and making it faster accessible is really critical in order to make everything go faster and work better. But the cool thing, and again, you said you don't know when this podcast is gonna air, but today's August 4th. And on Monday, August 7th at 10 in the morning, there's gonna be a really awesome podcast where you're gonna hear from 16 different Intel customers. And the names are pretty awesome. I mean, you're gonna hear amazing things that this technology is doing in e-commerce, in healthcare, across storage OEMs, in cloud service providers.

You know, this technology as an SSD, as a faster SSD being used either as storage or being used to extend memory for some applications is absolutely gonna take off in the market. There's no doubt about it. Interesting enough about the Wikibon report, they were specifically comparing an Optane SSD, you know, a hard drive form factor, to other form factors. And I think they even ended with, we don't know if Intel has any plans to do a DEM or a memory like that, but we already know that's actually coming.

That's what was demoed there. So from my perspective, faster hard drives, awesome. But the real value is when we can change our application to interact with persistent memory. And that's where I see something like this actually drastically changing things. Well, not only that, I think the ability to change the application to interact with persistent memory that strips away all of the abstraction that the OS puts to accessing a hard drive. All of that legacy stuff that's in the OS that's designed for SSDs, and even for some newer, I mean, for spinning rust and SSDs.

Once you strip all of that stuff out and make this stuff PACE addressable and the application can actually use it directly, that's going to be, I think, pretty surprising and game-changing for a lot of applications. I agree. I think that future, that's a lot of work, right? Just like when you have to refactor if you're gonna take advantage of the cloud, I think this is a similar kind of work effort. Well, so I love the fact that you guys are excited about that.

And I'm absolutely excited about that too. I'm kind of surprised two and a half times the number of VMs per server isn't enough. I mean, you guys are really getting dull to how exciting this stuff is. Well, I was gonna make mention of that two and a half times the VMs per server, which is legitimately a huge value. So you know what I'd be interested to see is, as our buddy Howard Marks will tell you, most performance testing is pretty terrible.

I would like to see the details of that, how deduplicatable, how compressible, how reusable, and the details of the performance. I think that's gonna tell the real story. Yeah, I think the evaluator group on their website has a lot of information on all of their testing methodologies. I would definitely encourage you to go check it out. So someone sent me the link to that a while back ago and I looked at the high level results and I said, you know what, I'm not surprised.

There's two elements on, if you look at like a modern day E5 or even E7 processor, how many times, Mark, have you gone into your data center and looked in vCenter and the bottleneck for your virtualization platform is CPU? Never. Once, once. Yeah, once. And that's maybe a high- It exactly was. It was some crazy analytics. So when the Twitter users said, you know what, Keith, here's the results with Intel Optane, and I said, you know what, I'm not surprised.

I tell customers all the time, buy as much memory as you can afford and then ask for more money for memory. And because the faster your IO to that memory piece is gonna be, the better. Then the second layer of that is storage. Buy the fastest storage you can buy. And the details of what we're learning about Optane and the speed improvements and the sheer amount of bandwidth from the NVMe, the ability just NVMe provide more bandwidth to faster drives. Yeah, I would expect an incredible uplift in a number of VMs that you can run on a system.

5 times number, but not surprised. Well, that's good. You guys are hard to impress. It is an impressive number. It's just not a surprising number. If you've been around this a long enough, you know where, you know, if I could put a competing technology, if I could put DSSD, the defunct DSSD, I don't think it's a defunct, but if I could make that a storage object, I'd expect that those types of performance increases in a VMware cluster, if that product was positioned towards this use case.

It's not, Optane is. I mean, absolutely. You got to think about the cost of that sort of a deployment where this is, you know, 10% of your overall capacity is Optane. The rest of it is Intel 3D NAND on our P4500 NVMe based SSDs with the 3D NAND where the costs are just coming way down in dollars per gigabyte. And you put the two together with the latest CPUs and you get a tremendous benefit. So for all the developers out there, people who are thinking about, man, rewriting my app to take advantage of persistent memory, that's a daunting task, or it's going to sound daunting to them.

What can they do now to prepare for that future? Oh, absolutely. So we've been working actually for the last, I don't know, roughly three years with the Storage Networking Industry Association, the SNEA group, to standardize a set of persistent commands. And this concept of NVDIMM that you've probably heard of has been kind of also helping to pave the path. So it's not as though people are starting from scratch today on this infrastructure. And as you saw, SAP, monster of a database, they've already shown proof of life of it working with the memory persistence in this DIMM form factor on stage.

So it's not that far away. Yes, this is a daunting task, but we've been working on it for three years and we're going to continue to work on it. But if you haven't started working on it, holy cow, maybe you ought to look to get acquired. If you have started working on it, just keep working. We've got all the connections within the industry. We are a big company here at Intel and we're helping out the majority of the major suppliers out there.

And we're definitely starting to seed the market. You can get ahold of an Optane SSD today and you can look at just paging and your paging performance if you're going to page out of memory and into this storage device. And you can see just what those kinds of benefits can bring you. As an example, VMware has got a host to swap cache paging feature. So if you need more memory in your VMs, you might not have to add more memory or you might not have slots for it.

Put in a couple of Optane SSDs and enable that feature and pipe it over to those Optane SSDs. You're really going to be pleased with what you see. All right, with that, today was national cookie, chocolate chip cookie day. So if you missed it, sorry, you have to go back on your diet. But that's okay. Take this podcast, go into the kitchen, make some chocolate chip cookies from scratch. I have a great recipe. Hit me up on Twitter.

I'll give that to you. And you and your loved ones can learn about Intel Optane and eat some delicious cookies. With that, Mark, where can people find you on Twitter? You can find me on the Twitters is at Cincy Storage. And James, where can people find you and more about Intel Optane? So they can find me at at doboysmeyers or look for the Intel SSD guy. com forward slash Optane. com. Subscribe to us, rate us in iTunes.

Talk to you guys next episode of the CTO Advisor.