Telemetry Data in the Public Cloud
Transcript
>> Hey, it's Keith Townsend. Principal of the CTO advisor. And thank you for joining us for a another conversation with an Intel subject matter expert, sponsored by Intel. We have just Josh Hilliker, solutions architect, actually cloud solutions architect, and that's important because we're going to get to this. Why this title is specifically important as we're continuing conversation around the CTO advisor hybrid infrastructure. If you remember, we extended the private cloud, our private cloud, or private data center out to the public cloud to both AWS, Oracle Cloud Infrastructure and Google Cloud Compute.
But we've run into this kind of conundrum as we've expanded out to the public cloud. We're kind of lost, when it comes to telemetry data we're used to pointing our V realized instances at the hardware underneath in our data center and we're able to get this rich data set. Now we're worried about concepts such as noisy neighbors application even application performance, et cetera. So, you know, I kind of pushed our friends at Intel to find out what can we do with telemetry data.
If we're all Intel in our private data center and we're all Intel in the public cloud kind of back of the napkin engineering would tell us that we should be able to leverage the platform across public and private cloud. Well, I'm here to tell you Josh is going to talk to us about how we do that. Josh, welcome to the program. >> Thanks Keith. Thanks for letting me join today. >> So, you know what, I've been searching for this Holy grail until our conversation last week about this concept of telemetry data, obviously the private cloud or public cloud, there's this concept or perception that they hide their underlying infrastructure from you.
They normalize it out so that x86 is x86 and you just don't get the insights. One of the things that my customers have always complained about is the lack of insight into the public cloud physical underlay. Talk to me about the capabilities that Intel enables with their public cloud part. >> Absolutely. And what a great topic, Keith. We talk about telemetry and being able to get the true hybrid Holy grail, right, of show me telemetry I have on prem on, you know in the cloud and being able to look at it across equally.
So for me, from a telemetry standpoint as you look at, you know, the main cloud providers, you're absolutely correct. Some of the lower instances, don't give you that level of insight into the hardware, but what I'm here to share with you today is that on a number of the instances you can get the specific Intel performance monitoring units from those platforms. And so let me talk a little about what is an IPU and start to kind of, or a PMU, excuse me sort of walk you down that path.
So a platform monitoring unit is access to, you know the MSRs it's access to the perf-driver to get different counters. And those counters give you super cool things like the ability look at cycles per instruction. So how is your app running? How is your infrastructure running? Is it efficient from a cycles per instruction or not? We look at things like utilization. So what is it truly doing on the utilization front from the CPU? What frequency am I getting?
So say you're and not really to do a a compare against, hey I purchased X, am I getting X? It's more of like how is it truly operating with my workload on the box? Am I getting the frequency I expected? So when I do a comparison of on-prem versus public cloud am I seeing the same frequency or I have issues? And the last area of super, like as we talked about a week ago that it just excites the heck out of me is team app.
So top level micro architecture a model where I can actually dig in and look at where's my bottleneck happening through my application. So you could run it against your app and see, okay where is it? Is it back inbound, front inbound? Am I having, you know am I having issues where it's retiring incorrectly or it's making a false assumptions? What is actually happening through the team app tree? So that was I just a quick thing on just what does a PMU, why does it matter?
This is the like very rich Intel telemetry that usually we talk about on-prem that now when we get to cloud, it's like, yes. And I would love to share with you what those instances are and then what I conditionally can do with them next. >> So actually that was my next question, because when I think about that capability, I would imagine, of course I should be able to do that with bare metal but these are not just bare metal instances as I understand what instances can I do this with, lets focus on AWS?
>> Okay. Got it. So currently today, and I knew this because I tested it. So not just am I giving you the, hey, here's what I heard. No. Actually last, what's funny enough, the last, you know month and a half, I've been testing on AWS instances to do this very thing, to make sure I can get the data that I'm talking about today. And I mentioned this in AWS reinvent course, gosh, a couple of weeks ago where I actually shared this information back with AWS community, but here are the five or the three big categories and some of those sub instances.
metal of course. So those instances are good. You can get all the PMU data I was talking about. Let me be laser clear for those folks that are just... >> (indistinct) >> Say again. >> Those are computers every C5 is the connotation is mainly because compute CPU. >> Yeah. Yap. Correct. Sorry yes. metal. So all those instances will give you everything I'm talking about, PMUs but there's one for the folks that are kind of what I'd say my fellow hardcore telemetries, let me be clear on what when I talk about PMUs, there's three different categories of PMUs.
There is core, off-core and on-core. And what we're, what I'm talking about today is really the core and off-core. So I can get the team app, the CPI utilization frequency. Some of the stuff I can't get is just the Newmont stuff. And of course, that's for obvious reasons of how the instance is set up. I won't be able to grab some of that on-core pneuma information, but that's okay. You get still get a lot of rich data with the PMUs.
>> So let's bring this conversation to a little bit higher level, basic application performance. One of the fears that I have when I place an application that wasn't designed for a public cloud is that I'm not able to troubleshoot performance. So let's take a simple workload or not so simple workload of Microsoft Seat. When I'm in my private data center and I'm having some SQL performance issues. One of the things that I can look at is CP cycles, or what is it, the measurement of the instruction per cycle.
>> Josh: Cycles per instructions. >> Yeah, cycles per instructions. How can I use these PMU dongle inside of AWS to troubleshoot let's say a sequel query that's running slowly. >> Yeah absolutely. So the same method applies on using the perf-driver in the operating system as part of that repo to the run the same exact, you know, telemetry whether you're using something like a cAdviser or using containers or views and that collect D on it or if you're just doing a batch processing using one of our Intel tools called Intel Plum tree Collector you can actually scrape that data out from the instance and do that exact same type of analysis.
Now there's two parts of it that I want to have an answer to. One is yes, I can use CPI as a kind of a initial, you know, comparison to the two. But really when you get into that team app is when you start to really get an idea of where is the problem. Is the CPU bound memory bound? Where is it happening? Kind of where in the code is it getting slow? So for example, gosh, about five months ago some of the code that I wrote, I was having some issues and I said, all right, I'm just going to go after this from a team app perspective and figure out what it was.
And I figured it was CPU bound. And I was asking for the data was creating CPU issues. And so I started to kind of look at, okay how can I reformat my code? What am I asking for it to do? And kind of reworked my scripts to work a little bit more efficient. >> So one of the things that I'm really excited about is data center orchestration across private and public clouds. And now specifically as the CTO Advisors hybrid infrastructure is spanning both private data center and public cloud.
I will love that ability to just under feed these orchestration systems, telemetry data. So I can make workload placement decisions. I'm not a big multi cloud person but there are instances where I would want to say this workload is best suited for this type of instance. And here's the proof of it via the telemetry data. We as Intel we're working with the orchestration community and allowing this type of capability. >> Now absolutely. So one of the, you know, the top orchestrators and schedulers out there is Kubernetes.
And with Kubernetes, we've been working heavily on providing additional capability into it for detection of what the platforms are. So one of the key features is node feature discovery and it allows you to basically on the nodes figure out what do they have, what features are available or what features are being published to the system. So some of those, some, you know some of the lower end systems you will be able to grab certain data out of but some of the systems I talked about you can grab the features that it has available that you can leverage that way.
When you set up a workload in your workload template, you can say, all right I need a machine that has AVX 512. I need, you know, so beyond the classic, you know I need this amount of CPU, this amount of memory this amount of disc space I can go deeper now with features. That's all node feature discovery. That's integrated into Kubernetes now, which is really cool because then you can start to look okay what's where do I want the workload to go based on feature?
And then the other part of that is the extension we made to cAdvisor. And so with cAdvisor, so it's Google cAdvisor, it's there, you know they're a container architecture container monitoring that they they push externally. We've made a major add on that just went in over the last two months into the, you know the master where it'll use perf-driver to grab all that rich telemetry out within the container. So then you can do kind of you can start to marry up, keep the concept right.
Of how is it performing and then what is the workload need? And then start to make placement decisions around that. >> So let's zoom out. I did a video recently where the folks at Twitter and Intel talking about Intel's networking and ADQ kind of an adjacent concept ADQ uses telemetry and the ability to control data lanes to kind of guarantee performance. So I originally did a video with Intel and Twitter about Intel networking ADQ. Basically the ability to create dedicated lanes of IO for the most critical traffic which reduces overall CPU usage and overhead and optimizes the system.
Twitter made some really impressive claims about how they were able to reclaim CPU usage in patients. And I think ultimately that's my ultimate goal when I'm thinking about a distributed system. And when I'm thinking about telemetry data I know we're not to the point where it's all networking specifically is being used or exposed to but as Intel given thought to exposing, you know kind of all of that data between network storage compute even security as Intel expands into these other areas as core?
>> Oh, absolutely. I mean, when we look at the telemetry landscape, there are the, you know the key domains that we care about right. There are, you know there's the performance, there's the health there's the security and there's the power. And those are the key things. So when we look at performance, it's beyond just CPU and you, you called out beautifully around, you know we look at the network devices, we look at the smart Knicks we look at SPGA telemetry, right.
On what is it going to give me and how can I use it when we look at the hard drives a couple of different aspects, right? There's, you know, what can we get out of SSD? What can we get to MVMe drives? What can we get out of our PMM modules? And the telemetry continues to get richer and richer. And I'll give you one, let me dig into one area real quick. Well, actually let me go high, let's say high level Keith and then I'll drop down into a specific use case.
The goal is to get all that telemetry together, right? So with this concept called the modern autonomous data center which is have a modern operating system have modern hardware and being able to feed all the telemetry from those two together. So you can make the right decisions. Right workload placement, and actually have a kind of a workload strategy on what you're going to do. So from a, you know, there's workload of where am I going to put it but there's a strategy on, okay, I want to place workloads.
The, you know, from a security standpoint on-prem. I want to do it from a performance standpoint. I want it from a cost standpoint. I want to, this is a new R & D area. We're going to try what we don't want to build an infrastructure, whatever that strategy is you then map it to your placement, kind of your blueprint on your placement and saying, all right what metadata do I need? Or what telemetry do I need to make that decision?
So we're, we're opening up everything that's possible to make that call, whether it's the feature set, whether it's the health of a component, whether its usage whether it's the security information we talk about you know, Intel resource director technology. So you talked earlier about noisy neighbors. Absolutely. What data can I pull out so I can look at when there's issues. And if I'm seeing that, you know it's having to rewrite the cache what is it doing exactly from a security standpoint.
We can look at it from a noisy neighbors on making sure that we're allocating the cache correctly and that we're monitoring it, allocating it and making sure that the best workload gets it's, you know kind of it's better share or better usage of the hardware and software in question. So then from a, about to dig in just a little deeper on this, you know one of the things that I've been excited about over the last year as we've been rolling out PMM, so our persistent memory module, right.
It's super cool. We're given dims and they do have a different couple of their features. I'm not going to go to that kind of marketing spill but it's like the we've extended our SSD. Sorry our NVMe drive kind of philosophy on telemetry to PMM. So you have things like a health analyzer that gives you health on the dim. And I just, I can't emphasize enough how super cool that is because when you know the health of a dim you can start to track and know when you got to replace it.
And today that's one of the things in like the normal dim market that we just don't have the level health data, right. I've got to make, we've got a full machine learning module on how to take the data and then start to learn from it to get a health score. But in our payment, we got it. And our MVMe drives, we got a health score. So it's like the beauty is if I'm placing a workload I can say, okay, I want, you know, do I have also health requirements on that workload?
Like, I want to have a system that is at 90% health and drive 90% health at PMM. I wanted to have, you know, X amount of cores available. I wanted to have AVX 512. I mean, you can start to lay out that metadata and really start to use every possible, you know compute cycle and every cycle within your infrastructure. I think that's what you talked earlier. That is the goal, right. Is I want to use everything at at a hundred percent that I possibly can whatever that a hundred percent means.
Like you can say, okay, I'm at 80% is my a hundred percent, great or I'm truly going to tax my infrastructure and get up to a hundred. >> So I really appreciate this conversation. We can go much deeper. We can talk about the open infrastructure and the ability for its OEM partners to tap into this capable into the modeling capabilities. We can even talk about entails competitive edge. when you introduce the concept of PMM and how, you know PMM only works with Intel processors to date and how Intel is able to then again expand out to is networking solutions around 82 PMM optane storage and engineered solutions both in the private data center public cloud that compliment each other, and, you know kind of speaks to those five pillars.
You don't, you won't have to wait too long to have some of these conversations. We are going to have one of Intel's OEM partners talking more telemetry data, more practical. We talk theoretical today. And our next conversation, we're going to get down into kind of what tools are being deployed at the edge. com is the website where you can find this and more entail related content around the CTO visor hybrid instructor. Talk to you next CTO does.
