Double Dose of Kubernetes From Kubecon 2019 - CTO Advisor 109

Machine-generated from the episode audio and not hand-corrected, so names and technical terms may be imperfect. The audio is authoritative.

Hey, welcome to episode 109 of the CTO Advisor podcast. This is a double dose of the CTO dose on the CTO Advisor podcast. If that didn't confuse you, hopefully you'll stay with the two conversations we're having. One conversation was with Joe Beda, Principal Engineer with VMware. Joe comes to VMware via the acquisition of Heptio back in 2019. The second conversation is with one of Joe's colleagues, Brian Lowes, who's a Senior Staff Engineer with VMware, part of the Heptio acquisition. This is a really, I think, down to earth set of conversations around what VMware is doing to make Kubernetes more consumable and typical enterprise customers, specifically those VMware customers.

And then the conversation with Brian is more about the culture shifts needed to accept Kubernetes, both from a IT infrastructure perspective, and then from a project culture and the project accepting enterprise IT culture as part of project governance. Hope you enjoy the conversations. Hey, how's it going? It's Steve Townsend from the CTO Advisor. I'm in 64 degree weather, which is crazy for this time of year for me. I'm from Chicago, Joe. It's raining and everybody's freaking out here. You know, I saw a guy who was nice and bundled up.

The umbrellas. It was crazy. You're from Seattle. Yeah. So, you know, I'm like, this is just another... What is it? Wednesday. I don't even know what day it is. So we're at KubeCon 2019. I have Joe Beda, Principal Engineer over at VMware. So we're just going to call you Guru All Things Tanzu at VMware. So this is a super appropriate, I think, interview. KubeCon 2019. This is my first KubeCon, amazing 12,000 people, over 200 sponsors.

This is Kubernetes growing up. It's been fascinating over the last, you know, five years to see, you know, so Kelsey talked about in one of the keynotes, some of the very earliest meetups, just to see the growth of the community and, you know, how it's changed, how it's evolved, how it's gotten more relevant, more commercial. I still think that there's a heart to it, though. Hopefully that comes through and you've seen some of that. And I think for me, that's the energizing thing about coming here to KubeCon.

Well, I actually talked to Brian Lyles, who's also a VMware staff engineer, but also more importantly, a senior staff, by the way, a senior staff, which is a big deal. Yeah, no, I mean, like, yeah, no, I was, I'm a former VMware guy, so it's a big difference between staff engineers, senior staff, you know, Brian's knocking it out of the park. Yeah, he's knocking it out of the park. And I talked to him yesterday about the complexity of taking something that has a lot of heart and then bringing the enterprise along with it.

It's a delicate dance. It's a delicate dance. Yeah, yeah. So I felt, I definitely felt the heart of kind of the open source movement yesterday. And you're in a unique situation because, you know, you're one of the founders and creators of Kubernetes. You kind of created this thing that was Google scale and helped Google set a set of challenges. And now you're bringing that to the enterprise. So it's not only the technology challenges you're dealing with, it's that cultural shift of bringing what was a thing that was created in a valley and created to solve valley problems to, I think, the most traditional of enterprise customers, VMware customers.

Can you talk a little bit about that? Well, I honestly think that this is just the story of cloud in general. And I think people view cloud from really different directions. I think there's this infrastructure first view, which is really about OpEx versus CapEx. You know, can we outsource essentially hardware ops, right? And I think that, you know, a lot of people are viewing cloud through that lens. When you look at it from sort of the application developer, you know, application team and developer point of view, what it really is, is it's about taking the, you know, the cycles to turn the crank and making things faster.

So moving from like, you know, so virtualization took us from, like, I need resources. It's going to be months because I have to go and order stuff from Dell and get delivers and rack and stack, right? To all of a sudden now it's like, hey, you know, we have virtualization. I can allocate a server using something like vSphere. And now it's like maybe on the order of a week or two weeks, right? Which is amazing. Which is an amazing enabler, right?

With the advent of cloud, now it's down to 30 seconds, right? And so you see these phase changes as you go from, like, you know, three months to three weeks to 30 seconds or three minutes. Just from a personal perspective, I've given up my home lab. My home lab is AWS. I mean, I can literally turn it off and turn it back on as needed. And it has been from an ability for me to learn, you know, if I want to, now if I want to spin up Kubernetes, why would I, you know, why would I spin it up and why would I get three nodes, blah, blah, blah, and spin that up when I can literally have a cluster up and running in 45 seconds or, you know, whatever the time period.

Yeah. So, yeah. So I think exactly right. So that, like, for me, from the developer's point of view, cloud brings, you know, API for self-service and, you know, and elastic, right? You can sort of, you know, and the cynical sort of developer in me is like, it's not talking to a sales guy as a service, right? It's this ability to just get your job done without talking to folks. And I think what one of the, you know, when we look at Kubernetes, you can look at it from either of those directions.

On the one hand, it's a way to actually start providing efficiencies and, like, get more out of your resources, that whole thing. But then there's another side of it is, how do we provide a cloud-like experience, API-driven, self-service, and elastic that is universal and runs everywhere? And so I think so many folks spend a lot of time moving to cloud. They look at it from the infrastructure point of view, and then they find that they don't get the acceleration on the application side, point of view, without actually embracing that sort of cloud-native way of building and managing and driving applications.

And so I don't think there's a Kubernetes problem. This is a cloud problem around how do application teams learn to modernize to take the most advantage of this API-driven, self-service infrastructure. I'm on record saying, I think the Kubernetes thing is going to take care of itself. From a... How do you... Like, Kubernetes right now is really hard, really complex to manage. VMware and all of your competitors and partners in the ecosystem, they're making strides to make that go. We're working hard on it.

I mean, like... And a lot of that's happening in the community also. I think a big part of our thing is that we think the easier we make Kubernetes for everybody, the better it's going to be for everybody. Right. I absolutely believe that that's going to happen. We actually sponsor OpenPaaS, the OpenPaaS homepage, because we believe once Kubernetes is to a point, it's where the puck is to these functions as a service, et cetera, which is what I wanted to talk to you...

Another thing I wanted to talk to you about, which is that developer experience. Everything from giving developer a virtual machine, or bare metal, to a virtual machine, to a container, to now this environment in which it's API driven. When you look at the cloud providers, what's appealing now is not just I get some IS resources, but I get some tools that get me 80 to 90 percent of what I need to get there from these other services. How does VMware view kind of this other part?

Once we've kind of taken care of the uninteresting parts of providing resources, what's the next level in VMware's vision? At the end of the day, it's all about making developers be more effective, ship software more reliably, faster. I hate to use the buzzword, but so much of digital transformation is really about this... I'm naming names here, but don't take them literally. It's the Teslas of the world learning how to become a car company, while the Fords of the world are learning how to become a software company.

That dynamic of software going from being a necessary evil of doing business to something that is a differentiator and a necessary thing to succeed in the market, that change is happening across every industry. I think that our goal with VMware is to actually provide a modern platform such that the IT departments can enable their development teams, their application teams, to move at the speed that they need to to be competitive, while also being respectful of both sort of the existing investments, the existing rules, and the complex landscape that enterprises deal with that oftentimes doesn't apply to smaller companies.

Let's talk about how VMware is helping kind of bridge the gap between the raw... I got exposed to the raw CNCF this week. Like this is... Whoa! I got... Oh, who knew Linkerd was Linkerd D? Just these nuances that you don't get on Twitter, et cetera. One of the things that is painfully obvious to me that a lot of the Kubernetes and open source community, they weren't born of enterprise IT. They were born in places that move fast, they break stuff.

If something's working today, great, we'll use it. If it's not great, we'll eject it and use something else. Or fix it. Or fix it. We can... It's kind of like if you have a car in Havana. You're not going to get new parts for it, but hey, you know what? We're going to make it work and it's going to look great. So same culture. I love that. That culture doesn't exist in an enterprise, but when I look at the stuff that they're doing with some of the companies...

I talked to a vendor yesterday, Mobile Edge X, who's tying 5G and containers together, allowing game developers to be able to scale... Being able to scale instances where players are at. So if you're doing Pokemon Go and there's 100 players that show up, wow. You can scale up 100 containers there. The enterprise needs that. We need companies like VMware to show us the way to kind of be the gateway to this community. Because I can't see the enterprise consuming CNCF projects natively.

What do you see VMware role in that relationship? One of the things that influence Kubernetes, but I think also influences the CNC, the cloud native ecosystem as a whole, is a lot of times what we call Unix philosophy. It's this idea, when you're learning the command line on Unix and you understand the subsystems, it's a bunch of small things that are relatively simple that compose together to create something more complex, something more interesting. That is, as opposed to, I think, more traditional things like...

I don't want to pick on Windows, but Windows, it's a more all-in-one experience. I think one of the reasons why we've seen, at least on the server side, Linux and the Unix world win out over Windows is because that philosophy is incredibly flexible in the face of new challenges over time. There's more to wrap your head around. There's more for you to understand, but also it can sort of bend versus break as you throw new challenges at it. I think that one of the reasons why, amongst this set, we've seen things evolve is that having all these different projects creates an enormous resiliency that you don't get when you have a much more vertically purpose-built type of product.

I think what we're going to see now, and I think this is VMware's point of view, is that we want to walk the line where we help people view this as a whole. This is why we introduced this new brand, this new effort called Tanzu, with the idea of let's view this as a unit. Let's view this as an experience where we can help folks get started. We can help them make sensible choices for them out the gate. But then as they hit limits, they can actually then go deeper and start taking advantage of the sort of beautiful chaos that is the cloud-native landscape that we're in right now.

I love that approach. I think Tanzu is kind of this umbrella concept of if you want a safe, enterprise-friendly way to approaching this, Tanzu is the entry point as you kind of get your ... Because this is day two for me, and day two, I'm starting to see how these pieces put together. And there's gaps, and it's rough, and it's like, yeah, it's not perfect. The open tendency is the problem. There's a lot of things that need to be solved.

But once you've kind of gotten your feet wet, there's absolutely ... It's balanced. The options are balanced. And I think it's painful to go through. I'm not going to lie about that. It's painful to go through, but I think at the end, it'll be worthwhile. A couple of trends that we do see here is that we do see that there are some enterprises that are further along this journey than others, and they are embracing becoming a software-driven company. And their relationship to the software changes.

Oftentimes they're not just consumers of these projects. They actually turn around, and they actually start contributing. And so if you walk the floor, you'll see companies out there that are end users wanting to be involved, wanting to actually be part of the community. Yeah, I saw maybe about six or seven end user sponsors. I have to absolutely check by ... I think Home Depot is one, a couple others. I'm anxious to see kind of why they're here and what they're doing.

Right. And then I think the other thing that we see is that enterprises are not homogenous. And so you'll find that there's going to be certain parts of the enterprise that are like, hey, they're doing relatively straightforward stuff. This opinionated stack for getting started works great for them. But then you may have a group that's doing really interesting forward-looking stuff, doing a lot of machine learning, doing some big data analytics, doing some really interesting stuff on the edge. Those are the folks that are like, hey, this is a great starting point.

How can I break the rules? How can I actually start leveraging the rest of the ecosystem in a way where they're not throwing the baby out with the bathwater, and they're not totally starting over as they need to actually do things that are a little bit off the beaten path? So Joe, we could talk probably for another half an hour on this topic, because it is an amazing time to be in IT. I really appreciate you taking time in your busy schedule.

Kind of like this is ... VMworld is my conference, and you're a VMware employee. But I think it's safe to say that KubeCon, CNF ... What is the CNF part of it? CNF? CNCF. Cloud Native Computing Foundation. Cloud Native Computing Foundation. Yeah. This is, I think, to say this is your show. I appreciate you taking time out of your schedule. Well, thank you so much for having me, and it's great to see these worlds converging.

It's been wonderful. Well, that's it for this episode of the CTO Dose One Site at KubeCon 2019. For more coverage, make sure to subscribe to the YouTube channel, LinkedIn, and Twitter at CTO Advisor on Twitter. Talk to you next, CTO Dose. Hey, how's it going? Hey, how's it going? It's Keith Townsend from the CTO Advisor, and we're still here in beautiful San Diego, KubeCon 2019. I'm joined by co-chair of the KubeCon, and I like to call it KubeCon because it's Kubernetes.

But evidently, I've been corrected by the 12,000 people that have already been part of this community. It's KubeCon, so I'll submit. And senior staff engineer at VMware, Brian Alton. Brian, welcome to the show. Yeah, thank you for having me. So right before we get started, we've been back and forth just on Twitter and in person about the culture of KubeCon and the difficulty of integrating the two different audiences for a super technical show, but super critical. I mean, it's over 200 sponsors, and you walk the show for us in two different spaces, 12,000 people.

This is the real thing now in this year. Definitely. I haven't been to one. People told me last year was great, but for me, this feels like a enterprise conference coming from the outside. But to some of the more traditional CNCF crowd, et cetera, this looks a little bit different to them. Can you kind of use the parable you shared with me? All right. So what I was saying earlier is the way that I look at a lot of things, but specifically this whole merging of Kubernetes with the enterprise and the community that created it, the way I look at it is like this.

So let's say you have a person who comes up and they say, oh, it's a car. It's a red car. It's a fast car. And all those things are true about the car. But then you have a race fanatic or a mechanic, and they come up and say, oh, no, that's a 2018 Ferrari 488 Pista, and look, it's got the special insignia on the seats. They're talking about the same thing, but they're looking at it in different ways. And I don't know which side you put the enterprise on or which side you put the community on, we really need to understand that this is not, Kubernetes is not just a thing.

It represents a whole bunch of things depending on who you're talking to and the context that you're talking to them in. And I think traditionally, I've always looked at Kubernetes as, oh, that's a fast car. I looked at it as this thing is a way to package, it's a way to orchestrate containers and manage containers. But when you come into the keynote yesterday, it's, man, it's these other projects, Helm, XCD, Jaeger, which I found, I didn't know anything about Jaeger until I sat down with a guy who's like, man, it does these amazing things.

It is amazing. And, but that's not part of the core project. So when I get, you know, when I'm in my lab and I'm doing my kubadmin and kubectl and all these things, there's so much more to it. It's hard to, it's really hard for me as an enterprise guy to come in and get my arms around it and say, oh, this is what I need to start out with. And this is the different pieces. I love the analogy that everyone has used, that it's all very, very small parts that come together, kind of like Lego or Minecraft.

It's very small things that come together that form these complex things that do the complex parts. But in the enterprise, we haven't typically thought that way. You know, we get Windows, and Windows is this huge thing that does everything that we need it to do. And if it doesn't do what we need it to do, we have Windows. Right. So we've been through this before. Think about when Linux came and hit the, and Linux in the enterprise is a big thing.

No one even talks about Linux in the enterprise anymore. We just do it. So think about this. Linux came out. I was using Linux in the mid to late 90s and downloading it and having it configured. And it still took from the 90s to the mid 2005, 2006, when people were actually like, oh, okay, this is going to happen. Yeah, this is like a real thing. Yeah, but think about what Linux is. When we think about Linux, Linux is a kernel.

And then you have your distributions, whether it be the, you know, Ubuntu from Canonical or RHEL from Red Hat. And it took a long time for us to get to that place. But we always had Linux. And I look at Kubernetes the same way. We have Kubernetes. We're still trying to figure out, we understand that it solves our set of problems. And enterprises should be paying attention because it definitely solves your problems. We need to figure out how to package it.

And that's what a good distribution is. But this is different now because we don't want to make the same mistakes Linux distributions made. How many ways have there been over the years to package up a Linux application? And even still today, you have devs and you have RPMs and don't even talk to the Arch people or the Gentoo people because they have their own thing. So what we want to do in this cloud native and specifically in Kubernetes, we want to keep that Kubernetes piece pretty pure and we want to take a different method this time and we want to build on top of that, but at least keep the base of it pure.

So one of the things that's kind of unsettling is the speed at which it's happening. Like this is happening way faster than Linux did. I remember Linux in between the mid 90s to 98 when I first got into IT, proper IT job, and we were still compiling, you know, distributions or the kernel directly, etc. If you really want to play with it, there were people who were doing tricks like blowing up monitors by messing around with the video drivers. Now what's I think what scares a lot of enterprises is that you can get a enterprise ready distribution of Kubernetes and build some enterprise class applications, mission critical stuff on it.

And then the fear is that the underlying stuff just goes away from a specific project. Oh, I was using this project for messaging. Now this other project is now the message, the message du jour. How do you help enterprises kind of navigate that fear? Well, so the first thing I tell them is, OK, because if this is a simple answer, we would have been solved it. But a lot of the things that you're using now can be used inside of Kubernetes.

You might have to package them different. They might have different operational things that have to deal with. But what we're going to have to figure out as a community and as organizations providing software is that we're not just talking about open source. We're going to need real plans for basically COTS, coming off the shelf software. We're going to need real plans for services we don't provide, like what the ISVs provide. And that can't come until we have a good base.

And I think enterprises are basically trying to put the cart before the horse. And they're saying, well, we need all this before we even get to the base. And guess what? They're people. And I understand why they're doing it. But what we have to realize is that this can happen at the same time. And it is moving faster than anything we've seen before. But the process that we're using right now to build Kubernetes is making it fairly stable.

We're still getting new features. So we have to look at, instead of looking at next month or maybe even six months, we have to look 12 to 18 months out because we look 18 months ago and where Kubernetes was versus where it is now. Or if I go back 36 months and made some predictions, we're way further. So 36 months ago, Mesos was still a player. Dr. Swarm was like, all right, maybe this will work. And people were like, well, we're just going to run everything on Marathon or Aurora on top of Mesos.

What did Mesos do recently? They got Kubernetes projects. And we just have to realize that we have to start from somewhere and we're at the beginning. And you don't have to go wholesale on the Kubernetes right now. You're running your software. You're running it. You know, you have your vSphere clusters online. You have whatever cloud that you're using. You're doing that now. Well, guess what? You can continue doing that. But keep in mind that this Kubernetes thing will help you use your resources, whether it be memory, compute and network more efficiently.

And that's where we're moving to. And enterprise scale, what's another year? In enterprise scale, not that. So it's funny. In the Kubernetes world, years. It's 20% of the lifetime of Kubernetes. Yes. And enterprise scale, I'm still running HPUX. I'll still be running HPUX a year from now. So there's other reasons for that that's beyond technology. But a year is not. Well, we have to have a conversation of why you're running HPUX, though. It's all about whether or not the business wants to make the...

It's really interesting because it's related. So in this scenario, it was a health care organization. The application SAP validated on HPUX. It cost $10 million to revalidate that. Oh, that's fair. So do I spend $2 million to get off HPUX and spend another $10 million to... Or to spend $2 million to stay on HPUX or spend $10 million to revalidate it into $2 million to get off? So those are the types of challenges that meet head-on with enterprise.

And when we look at Kubernetes, we're like, oh, wow, I can't make that mistake of betting on a project that's going to go away. And I have these other processes that will stop me from moving forward. Right. But also, $1 million, $2 million, $10 million. It's not a lot. I mean... We're going over the rail like, but for this health care organization, not a lot of money. Yeah. Yeah. Yeah. So I think this conversation was spurred some really other interesting conversations about where's the real value.

And if we're pursuing value, then the Kubernetes journey, and not just Kubernetes, but this technology journey is worthwhile. Well, let me go on a little rant about Kubernetes value. Okay. This is assumed value or future value, or in some cases, current value. So imagine that you had a platform where you didn't have to optimize. You could optimize based on the application that was delivered. So instead of having to think about how many versions of this thing I'm running in production, your developers and your operations team can actually think about the things that make you money.

That is the whole premise behind Kubernetes. If we can abstract this running of software on one or more machines, and then make sure that we have things like service discovery, we can have file systems where they need to be. We can have the proper security. We can isolate things. Now that's actually the value of Kubernetes. Like I think sometimes we get lost in the concepts of deployments and staple sets and replica sets. I've had the same, I think I've had yet anyone articulate that to me.

And when I usually talk to Kubernetes, the reaction is typically, you know, you can do all these cool technical things and I don't care. I can do a lot of those things today. I don't really care about that. Help me understand the value. Yeah. I think you've articulated the value. No one cares. And that's what I think some people from my sphere of influence, that's something they get caught up in. If you have a software that needs to run and you want to scale it up and down, that's a deployment.

But you have software that you need to make sure that you run on every single node, maybe because there's some kind of scanner. That's a daemon set. And that's only important in the context of deploying the application. Sometimes the application developer shouldn't even have to worry about it. So in theory, we should be working on improving our stack, but instead of finding better ways to manage this YAML mess that people get lost in, why don't we just skip it? Why are we not thinking at a higher level of abstraction?

Because guess what? When you work for a bank and you have 1,300 or 1,400 or 5,000 dev teams and some larger banks do, they don't care about that. They care, am I making software? No, I am not losing money on the money that we bring in. We want to pay our dev teams, we want to pay them fairly, but we don't want to lose money on waste. And that's where I think a lot of people who are in the community of Kubernetes think now, but then whenever you're buttoned up against these CIOs, they're thinking at this other level.

They're thinking of, well, how efficient are we? And Kubernetes is a great tool for getting us there, but we have to make sure that we're building tools that allow us to get there and we got to start filling in that middle space because it's actually not Kubernetes. Now what we need is the grand scheme of how to build software on top of Kubernetes. I think you just answered my question of why I, that was the perfect argument against why I hate Kubernetes so much and why at the same time, I'm personally, my business is sponsoring OpenFast.

Why do I sponsor OpenFast, but I hate Kubernetes? I don't hate Kubernetes. I love OpenFast and I want that functionality. And I just, the Kubernetes stuff just needs to take care of itself so that I get that functionality and we can't get there until Kubernetes. Right. And then also, and then we don't help ourselves because there's a lot of board developers out there and a lot of really motivated people. And we get things like AWS released Lambda years ago now, and now they were first movers really in that space, kind of.

And then the whole ecosystem has followed suit. And I actually just had a conversation when I was walking here and we're talking about serverless and I said, why do you use serverless? Oh, you know, I can serve as functions and I said, all right, that's cool, but you can do that as an app. Yeah, absolutely. And I said, well, so really you just created something new. What's the good piece about serverless? I can tell you, not many people can answer that question, but actually the real power of serverless is something else.

The real power of serverless is having basically unbounded compute to do random things. And a good example would be something like what Capital One does. Capital One built this thing called Custodian. And what Custodian does is it cleans out their resources. It saves them money because they have so many accounts. And then it goes through their S3 and makes sure that things aren't unencrypted. That's the power of serverless. Now our actual compute can scale with whatever we want.

And the neat thing about Capital One that I can actually share is that they don't use serverless functions. They don't use it like that. And the reason why is because when AWS introduced Lambda and you had to use API Gateway, it wasn't encrypted. So you would basically have encrypted traffic, unencrypted hop, encrypted, and that didn't pass the auditors. So what I'm encouraging people to think about is let's think about if we're going to think about serverless and Kubernetes does allow us to do a lot of things like Knative and other things.

Let's think about instead of thinking about different ways to do what we're doing now, why don't we think about new ways so we don't have to do some of these things now. And that's what Kubernetes gives us the platform to figure out. So instead of just thinking of serving, think about, well, now I just have this huge batch processing thing that goes away when I'm not using it. Well, okay, well, hold on here. Batch processing, super simple. And now I don't have to spend any money on it when I'm not using it?

So we're getting to the, Brian, we can go on and talk about this. Obviously, we've gone longer than our typical CTO dose. Well, I'll have to catch Brian at a later date. Thank you for tuning in for the CTO Advisor, CTO Dose at KubeCon 2019. We'll catch you next CTO Dose, follow me on the web at ctoadvisor.