Telemetry Data in the Public Cloud

Machine-generated from the episode audio and not hand-corrected, so names and technical terms may be imperfect. The audio is authoritative.

Hey, it's Keith Townsend, Principal of the CTO Advisor, and thank you for joining us for another conversation with the Intel Subject Matter Expert, sponsored by Intel. We have Josh Hilliker, Solutions Architect, actually Cloud Solutions Architect, and that's important because we're going to get to this, why this title is specifically important as we're continuing the conversation around the CTO Advisor hybrid infrastructure. If you remember, we extended our private cloud or private data center out to the public cloud, to both AWS, Oracle Cloud Infrastructure, and Google Cloud Compute, but we've run into this kind of conundrum.

As we've extended out to the public cloud, we're kind of lost when it comes to telemetry data. We're used to pointing our derealized instances at the hardware underneath in our data center, and we're able to get this rich data set. Now we're worried about concepts such as noisy neighbors, even application performance, et cetera. I kind of pushed our friends at Intel to find out what can we do with telemetry data. If we're all Intel in our private data center, and we're all Intel in the public cloud, kind of back of the neck and engineering would tell us that we should be able to leverage the platform across public and private cloud.

Well, I'm here to tell you, Josh is going to talk to us about how we do that. Josh, welcome to the program. Thanks, Keith. Thanks for letting me join today. So you know what? I've been searching for this holy grail until our conversation last week about this concept of telemetry data. Obviously, the private cloud or public cloud, there's this concept or perception that they hide the underlying infrastructure from you. They normalize it out so that x86 is x86 and you just don't get the insights.

One of the things that my customers have always complained about is the lack of insight into the public cloud physical underlying. Talk to me about the capabilities that Intel enables with their public cloud partners. Oh, absolutely. And what a great topic, Keith. We talk about telemetry and be able to get the true hybrid holy grail right of show me telemetry I have on prem on, you know, in the cloud and being able to look at it across equal equally. So for me, from a telemetry standpoint, as you look at, you know, the the main cloud providers, you're absolutely correct.

Some of the lower instances don't give you that level of insight into the hardware. But what I'm here to share with you today is that on a number of the instances you can get the specific Intel performance monitoring units from those platforms. And so let me talk a little about what is an IPU and search kind of a PMU excuse me and sort of walk you down that path. So a platform monitoring unit is access to the MSRs. It's access to the perf driver to get different counters and those counters give you super cool things like people look at cycles per instruction.

So how how is your app running? How is your infrastructure running? Is it efficient from a cycles per instruction or not? We look at things like utilization. So what is it truly doing on the utilization front from the CPU? What frequency am I getting? So say you're and not really to do a compare against I purchased X. Am I getting X? It's more of like, how is it truly operating with my workload on the box? Am I getting the frequency I expected?

So when I do a comparison of on prem versus public cloud, am I seeing the same frequency of bad issues and the last area of super like as we talked about, we go that it just excites the heck out of me is T man. So top level microarchitecture model where I can actually dig in and look at where's my bottleneck happening through my application. So you can run it against your app and see, OK, where is it? Is it back inbound, front end bound?

Am I having, you know, am I having issues where it's retiring incorrectly or it's making false, false assumptions? What is actually happening through the tree? So that was just a quick thing on just what is a PMU? Why does it matter? This is the like very rich telemetry that usually we talk about on prem that now we get to cloud. It's like, yes. And I would love to share with you what those instances are and then what I can actually can do with them next.

So I think that was my next question, because when I think about that capability, I would imagine, of course, I should be able to do that with bare metal. But these are not just bare metal instances, as I understand. What is this? Can I do this with less focus on AWS? OK, got it. So currently today, and I know this because I tested it. So not just am I giving you that. Hey, here's what I heard now. Actually, the last what's funny enough, the last month and a half I've been testing on AWS instances to do this very thing, to make sure I can get the data that I'm talking about today.

And I mentioned this in a AWS reinvent course a couple weeks ago where I actually shared this information back with the AWS community. But here are the five or the three big categories and some of those sub instances. So anything in the C5 category around 9x large, 12x large, 18x large and 24x large and dot metal, of course. So those instances are good. You can get all the PMU data I was talking about. And those are going to be laser clear.

Those are the folks that are heavy on those, correct? Say again? Those are compute heavy. C5 is the connotation is mainly because compute. CPU. Yep, yep, yep. Yep, correct. Sorry. Yes. And then we've got M5, 12x large, 24x large, dot metal as well. And then R5, 8x large, 12x large and dot metal. So those all those instances will give you everything I'm talking about PMUs. But there's one for the folks that are going to get what I'd say my my fellow hardcore telemetries.

Let me be clear on what I want to talk about PMUs. There's three different categories of PMUs. There is core, off-core and un-core. And what we're what I'm talking about today is really the core and off-core. So I can get the TMAN, the CPI utilization frequency. Some of the stuff I can't get is just the NUMA stuff. And of course, that's for obvious reasons of how the how the instance is set up. I won't be able to grab some of that un-core NUMA information, but that's OK.

And you still get a lot of rich data with the PMUs. So let's bring this conversation to a little bit higher level, basic application performance. One of the fears that I have when I place an application that wasn't designed for a public cloud is that I'm not able to troubleshoot performance. So let's take a simple workload or not so simple workload of Microsoft SQL. When I'm in my private data center and I'm having some SQL performance issues, one of the things that I can look at is CP cycles per or what is it?

The measurement of the instructions per cycle per instruction. Yes, cycles per instruction. How can I use the these PMU data inside of AWS to troubleshoot, let's say, a SQL query that's running slowly? Yeah, absolutely. So the same method applies on using the perf driver in the operating system as part of that repo to run the same exact telemetry, whether you're using something like a C advisor of using containers or if you're using a collect D on it, or if you're just doing a batch processing using one of our Intel tools called Intel Telemetry Collector, you can actually scrape that data out from the instance and do that exact same type of analysis.

Now, there's two parts of it that I want to have an answer to. One is, yes, I can use CPI as a kind of a initial, you know, comparison of the two. But really, when you get into that T-MAM is when you start to really get an idea of where is the problem is a CPU bound, memory bound, where is it happening? Kind of where in the code is it getting slow? So, for example, gosh, about five months ago, some of the code that I wrote, I was having some issues and I said, all right, I'm just going to go after this from a T-MAM perspective and figure out what it was.

And I figured I was CPU bound. And I was asking for the data was creating CPU issues. And so I started to kind of look at, OK, how can I reformat my code? What am I asking for it to do? And kind of reworked my scripts to work a little more efficient. So one of the things that I'm really excited about is data center orchestration across private and public cloud. And now, specifically, as the CTO advises hybrid infrastructures spanning both private data center and public cloud, I would love the ability to just feed these orchestration systems telemetry data so that I can make workload placement decisions.

I'm not a big multi-cloud person, but there are instances where I would want to say this workload is best suited for this type of instance. And here's the proof of it via the telemetry data. Where is Intel working with the orchestration team and allowing this type of capability? Now, absolutely. So one of the top orchestrators and schedulers out there is Kubernetes. And with Kubernetes, we've been working heavily on providing additional capability into it for detection of what the platforms are.

So one of the key features is node feature discovery. And it allows you to basically on the nodes figure out what do they have, what features are available or what features are being published to the system. So some of the lower end systems you'll be able to grab certain data out of. But some of the systems I talked about, you can grab the features that it has available that you can leverage. That way, when you set up a workload in your workload template, you can say, all right, I need a machine that has AVX 512.

So beyond the classic, you know, I need this amount of CPUs, this amount of memory, this amount of disk space, I can go deeper now with features. That's all node feature discovery that's integrated into Kubernetes now, which is really cool because then you can start to look, OK, where do I want the workload to go based on feature? And then the other part of that is the extension we made to CAdvisor. And so with CAdvisor, so it's Google CAdvisor, it's their container monitoring that they then push externally.

We've made a major add on that just went in over the last two months into the master where it'll use perf driver to grab all that rich telemetry out within the container. So then you can do kind of you can start to marry up the concept of how is it performing and then what does the workload need and then start to make placement decisions around that. So let's zoom out. I did a video recently with the folks at Twitter and Intel talking about Intel's networking and ADQ, kind of an adjacent concept.

ADQ uses telemetry and the ability to control data lanes to kind of guarantee performance. So I recently did a video with Intel and Twitter about Intel networking ADQ, basically the ability to create dedicated lanes of IO for the most critical traffic, which reduces overall CPU usage and overhead and optimizes the system. Twitter made some really impressive claims about how they were able to reclaim CPU usage and efficiency. And I think ultimately that's my ultimate goal when I'm thinking about a distributed system and when I'm thinking about telemetry data.

I know we're not to the point where Intel networking specifically is being used or exposed to, but has Intel given thought to exposing, you know, kind of all of that data between network storage, compute, even security as Intel expands into these other areas as core? Oh, absolutely. I mean, when we look at the telemetry landscape, they're the key domains that we care about, right? There are, you know, there's the performance, there's the health, there's security and there's the power. And those are the key things.

So when we look at, you know, performance, it's beyond just CPUs. And you called out beautifully around, you know, we look at the network devices, we look at the smart NICs, we look at FPGA telemetry, right? And what is it going to give me and how can I use it? When we look at the hard drives, a couple different aspects, right? There's, you know, what can we get out of SSD? What can we get out of NVMe drives? What can we get out of our PMM modules?

And the telemetry continues to get richer and richer. And I'll give you one. I mean, let me dig into one area real quick. Well, actually, let me go high level, stay high level, Keith, and then I'll drop down into a specific use case. The goal is to get all that telemetry together, right? So we have this concept called the Modern Autonomous Data Center, which is have a modern operating system, have modern hardware and being able to feed all the telemetry from those two together so you can make the right decisions, right workload placement and actually have a kind of a workload strategy on what you're going to do.

So from a, you know, there's workload placement of where am I going to put it? But there's a strategy on, OK, I want to place workloads, you know, from a security standpoint on-prem. I want to do it from a performance standpoint. I want it from a cost standpoint. I want to, this is a new R&D area we're going to try, but we don't want to build on infrastructure. Whatever that strategy is, you then map it to your placement, kind of your blueprint on your placement and saying, all right, what metadata do I need or what telemetry do I need to make that decision?

So we're opening up everything that's possible to make that call, whether it's the feature set, whether it's the health of a component, whether it's its usage, whether it's the security information we talk about, you know, Intel Resource Director technology. So if you talked earlier about noisy neighbors, absolutely. What data can I pull out so I can look at when there's issues? And if I'm seeing that, you know, it's having to rewrite the cache, what is it doing exactly from a security standpoint?

We can look at it from a noisy neighbors on making sure that we're allocating the cache correctly and that we're monitoring it, allocating it, making sure that the best workload gets its, you know, kind of its better share or better usage of the hardware and software in question. So now from a part of digging just a little deeper on this, you know, one of the things I've been excited about over the last year as we've been rolling out PMEM, so our persistent memory module, right?

It's super cool. We're giving DEMs and they do have a couple of different features. I'm not going to I'm not going to go to that kind of marketing spiel, but it's like. The we've extended our SSD, so our NVMe drive kind of philosophy on telemetry to PMEM, so you have things like a health analyzer that gives you health on the DEM. And I just I can't. I can't emphasize enough how super cool that is, because when you know the health of a DEM, you can start to track and know when you got to replace it.

And today, that's one of the things in like the normal market that we just don't have a level of health data, right? I've got to make we've got a full machine learning module on how to take the data and then start to learn from it to get a health score. But in our PMEM, we got it and our NVMe drives, we got a health score. So it's like the beauty is if I'm placing a workload, I say, OK, I want you know, do I have also health requirements on that on that workload?

Like I want to have a system that is at 90 percent health, the drive, 90 percent health of PMEM. I wanted to have X amount of cores available. I want to have AVX 512. I mean, you can start to lay out that metadata and really start to use every every possible, you know, compute cycle and every cycle within your infrastructure. I think that's when you talked earlier, that is the goal, right, is I want to use everything at a hundred percent that I possibly can, whatever that hundred percent means.

Like you could say, OK, I'm at 80 percent is my hundred percent great, or I'm truly going to tax my infrastructure and get up to 100 percent. So I really appreciate this conversation. We can go much deeper. We can talk about the open infrastructure and the ability for Intel OEM partners to tap into this, into the modeling capabilities. We can even talk about Intel's competitive edge when you introduce the concept of PMEM and how, you know, PMEM only works with Intel processors today and how Intel is able to then, again, expand out to its networking solutions around ADQ, PMEM, Optane storage and engineer solutions, both in the private data center, public cloud that complement each other.

And, you know, kind of speaks to those five pillars. You don't you won't have to wait too long to have some of these conversations, we are going to have one of Intel's OEM partners talking more telemetry data, more practical. We talk theoretical today and our next conversation, we're going to get down into kind of what tools are being deployed at the edge, so to speak, to actually get these data insights. com is the website where you can find this and more Intel related content around the CTO Advisor hybrid.

Talk to you next, CTO does.