Replay: Is Kubernetes Right for Your Organization?

Machine-generated from the episode audio and not hand-corrected, so names and technical terms may be imperfect. The audio is authoritative.

All right, you're listening to and watching another great CTO Dose, CTO Advisor crossover episode. I have a good friend with me from the other side of the pond, Alex Ellis. We've had him on the program before. But Alex, you share some great news with me. What is that great news? Yeah, there was an announcement from VMworld from TechTarget. And there's basically every year a group of judges get together and they set up a whole bunch of awards and announcements, highlights of the show, winners in different categories.

And it turned out that, and you have to keep me honest here, OpenFaaS Limited got the best startup or featured startup and was a winner in that category. Now let's talk about the significance of this. OpenFaas is a Kubernetes related project. You guys work with more than just Kubernetes, which is really interesting. Is Swarm still supported in OpenFaas? This is something that I've actually been talking about recently quite a bit in my insiders emails in OpenFaas Slack, on Twitter, and really I think that Swarm slowly died two years ago and we all really need to be thinking about other options.

In particular, OpenFaas users, I would just expect you to be using Kubernetes. And if we hear this common argument, it's too complicated, it's too hard. Actually, we've made it really easy using K3S and using tooling like Arcade so that getting OpenFaas and Kubernetes are probably even easier than it was on Swarm. And once you have that, if you had a Swarm cluster in the olden days, yes, you could join it and create it and what have you, manage it relatively easily. But today we actually have managed Kubernetes services.

So those old opinions that were formed by people back then were when you still had to set up your own Kubernetes cluster, set up all the certificates, rotate them yourself and manage all of the nodes. Today with something like Amazon EKS, that has all gone away and you can get a very easy experience. So we'll get into kind of the integration of if you want to consume a functions as a service project like OpenFaas in a managed Kubernetes service, etc. But let's go back to this best of VMworld thing, because we've heard of VMware Tanzu and VMware's entry into Kubernetes, etc.

Help me understand, I am a judge for best of VMworld. So, you know, I kind of like, yeah, I kind of was surprised. No, I'm not surprised. I was one of the people that voted for OpenFaas to win the at-large startup or OpenFaas LTD or support company to win the at-large prize for best startup. But help us understand, what is the relationship between OpenFaas and VMware? Because I think I have to do mental gymnastics to kind of make the connection.

Yeah, I mean, it's a classic breakup story. VMware fell in love with OpenFaas. They were building a product on it. They reached out and sort of wooed me and offered to hire me to continue working on it at the company with a team. I went ahead and did that. And after 13 months, they were like, well, we think times are changing. We want to go in a different direction. And they sort of parted ways with me there. And then I've been out effectively on my own, keeping the project running, thinking of creative ways to build a business that can pay into this project, because it's a full-time job to maintain it.

And I used to have a team of four people working with me. Now I'm back on my own with the community. And GitHub stars don't pay the bills. End users very rarely contribute code or any money of any kind. But what I thought was quite sweet was that when I was at VMware, I created an add-on for OpenFaas, the vCenter connector. What that meant was when an event happened in vCenter, in a software-defined data center, it could fire off a function.

And we could remediate. We could audit. We could log. We could do whatever you wanted in PowerShell, in Python. It doesn't matter the language. So an example would be, and at the company I worked at before, VMware ADP, we had this classic problem. People are creating VMs on self-service. Nobody knew if they're being used or who they belonged to. Nightmare. You run out of your capacity. And the answer is, you send an email out to everyone, please, can you delete your VMs?

A couple of people do, but the problem's no better. You then have to buy another server and put it into your data center. And it gets worse and worse. So with the connector and OpenFaas, you can solve that problem very quickly by just tagging the VM when it comes up. And you know exactly who created it. So an event is created, and OpenFaas triggers off of that. OpenFaas function triggers off that event? Yeah. So that was pretty popular with pre-sales.

And with the office of the CTO whilst I was there. And then after moving on, they kept up their interest. They kept developing their ideas around it and rebranded the work that I put together as something called VEBA, or VMware Event Broker Appliance. And as some of us that are more hands-on will know, an appliance is effectively a pre-packaged VM with some software installed on it. And the software is OpenFaas, the vCenter connector, and I think recently they've also bundled in a reverse proxy as well.

So the upstream project plopped on an appliance, pushed in a registry somewhere. You can then download that and install it on your vSphere and start writing your own functions. And they provided the catalog. And in the beginning, they were quite active on Slack, lots of requests. Lots of asks. Can you add this feature? Can you do this code? Can you review this? Can you do the other? And that kind of dropped off and not really heard from them for about a year.

But they've continued what they were doing. And it looks like it's creating some value for them. So hopefully they'll come back and offer some support for the OpenFaas community. But I want to shift the conversation to a blog post you wrote a month or so ago about this decision matrix on whether or not Kubernetes is right for you. I'm just coming off of last night writing a blog post on do I still hate Kubernetes? And much of my frustration around Kubernetes, I think the industry has, the community has addressed some of that.

But first overview, you're in the Kubernetes community. I mean, you have KubeCon, you have a Raspberry Pi presentation coming up. You're running a open source, a couple of open source projects that are tightly aligned to Kubernetes. Shouldn't the answer to that question just be yes? I think, you know, I think that the temptation is for it always to be yes. And one of the things that I find in the OpenFaas Slack community and wherever I go, really, is often users will look at a technology, particularly if it's open source, because the documentation is out there.

The buying process has completely changed, but there's nothing to pay for. You're only paying your own time to set something up. They come in, they've had a look around, and then they think, okay, well, can I shoehorn in this problem and get it fixed with this project? And they say, right, yeah, I think I can, except X. And they come and ask you a deeply specific, extremely narrow technical question with no context and no insights into what they're trying to do.

And techie people will just answer them. And I've seen it. And it will be 90 replies in the thread. And I come in in the morning, and I'm like, what problem are you trying to solve? They'll say, and so we're looking at the wrong solution. Hmm. Whilst you see three, four of your dedicated, friendly community members have been thrashing around this idea, trying to solve it for them, they've not known what the problem was. And I see the same with adoption of Kubernetes.

People are adopting Kubernetes, but we don't know what the question is. And what this blog post does is it says, start by defining not only what is the problem, but what constraints do you have? An example we have here is, yes, a problem that could be solved by Kubernetes, but the constraints in place were, and it's actually a personal friend of mine, an ex-colleague. He didn't have any DevOps capability in the company at all. And he wasn't interested in paying for that.

He was VC backed, and he needed to be paying people to develop features. He wanted to defer as much as he could this idea of DevOps and paying someone for that, maximize the returns. Now, I thought that was a little bit sort of short-sighted to begin with. And then I saw there was a couple of quick wins. So he had one VM. He could have lost a lot of money. He could have lost the respect of his VCs if that one VM had died and the 200,000 customers couldn't get service.

That could be disastrous for the company. Right. Just mirror it. Job done. I mean... Not auto-scaling, not creating a resilient backplane, et cetera. Just mirror the VM, hon. And what about snapshots? He was using DigitalOcean. They have it built in their platform. You can set it up on a cron, so it runs every hour, every night. There's a snapshot of the database and maybe the deployment platform. Very cheap, very easy. And whilst hiring a Kubernetes consultant could have cost him tens of thousands of dollars, or they can pay a company like SciUp or Weaveworks and have plans like $1,000 a month per cluster.

He could have also just used the platform he had, mirrored the VM, put a load balancer in front of it, taken snapshots of the database. And that's what DevOps is about. It's not about shoehorning in a sexy technology. Or even if you really want to use it, or you've got someone inside the company that's desperate to try it out, it's not giving in to that too early. Sometimes it's appropriate, maybe most of the time. But there's always other options. Even managed containers, right?

Let's say Lambda wouldn't fit what he needed. Well, you know, you can use something like Google Cloud Run, and you can run an ad hoc container there. It scales for you. There's no billing idle. There's lots of different options. And it's not always self-managed Kubernetes. So I love that example of where Kubernetes isn't necessarily the right fit, or even the solution to the problem. But, you know, I kind of feigned this, you know, Kubernetes is too big, it's too complicated, et cetera.

But you've personally done a lot of work to make OpenFast consumable on top of Kubernetes for people who just need, they just want functions. And Kubernetes, I mean, OpenFast by itself isn't providing a runtime. So there needs to be a runtime. There needs to be an underlay for OpenFast to call. And you've kind of helped to package this thing in a way that it's consumable. Can you give an example of where you make, you've helped make OpenFast production ready for a smaller team?

Yeah. So there's a lot of requests that come through OpenFast Slack and GitHub. And you can always tell when it's being used commercially because they're very secretive about it. And they come in and they say, we are doing X. Here's our super narrow technical request. No context, no introduction. Can you fix it? Right. And then you have this long arduous process of winding it back and trying to figure out who is we? Are you working at VMware?

Are you working at NetApp? Are you working at a large bank? What problem are you trying to solve? Are you already using OpenFast? And this is very, very difficult. Unless you have a consulting relationship or they're paying for something like an OpenFast premium subscription where you've got a relationship with them, you have no obligation to give them enterprise grade support. And B, nobody in a community is under any obligation to look at their request or even speak to them.

The code is free. Professional services are not. And they're not included. Whilst we do everything we can to help people and generally welcome commercial users, I do think that having that relationship means that we can then help the company get success. The worst thing and the most frustrating thing is we're doing a proof of concept of OpenFast or within LATS. We're not going to pay for any consulting or any advice or any support, but we're going to raise all these feature requests.

We're going to bug you all the time on Slack. And eventually when we're running in production, we might come back and chuck you some sort of beer money. No, you will not have a successful POC that way. It's probably going to cost you a lot of time and energy and you might come to the wrong conclusion. For a small amount of money, you can actually partner with the people that created the project who have an ability to influence the roadmap and to respond to whatever questions you have in an expert way and get you further along.

Now, there was one company that saw the value of this and partnered with us over the course of last year. We hardened the OpenFast control plane for them, introduced the ability to run with gVisor, which is Google's isolation technology for containers. They were wanting to run multi-tenant code. They didn't want people to escape from the container, but they didn't have a way in OpenFast to set that value. So that's one of the things that we're able to do is prioritize that on the roadmap and also work through some other things with them where they had come up with a really convoluted system.

I was able to say, have you thought about X? And at the end of the day, they said, yeah, we've saved a lot of money, saved a lot of time by actually putting some money and funding up front for this. So I'm hearing two things. One, you should really know the problem that you're solving. You should always, both me and you are more than happy to offer advice in our overall disciplines, especially engaging customers and helping them along their journeys. POC is not free.

I wrote a blog post just last week talking about what the amount it costed us internally, the CTO advisor, something around $65,000 in labor and another $25,000 in services and expenses to run a comparison of the various VMware cloud solutions. This stuff costs money. And then the other thing that I'm hearing is that the direct input into the project itself. What were some of the learnings that you took back from that engagement and you've applied to the project? I mean, in my mind, as a IT infrastructure person, I get stuck in my head snapshots of the state of a project.

And I remember Kubernetes from last year when I wrote the blog post, why I hate Kubernetes so much, that it was just cumbersome, et cetera. But if I wanted to layer something like open, I really want open fast, but I look at the Kubernetes project and just think, man, Kubernetes is just way too heavy for me to get that functionality. How is that thinking a little bit outdated? Well, you know, it isn't necessarily. Adrian Kocroft is VP of, we're probably gonna get this wrong.

I think it's something like open source. Engineering or architecture at AWS. And he was on the software circus with Mark Coleman at Packet and they were talking about this. And he said, I really worry what you get when you come in and you layer down Kubernetes, Istio, Knative, and then your application code. You have so many layers of indirection and control plane. And if you just want to run a couple of functions, the cost per invocation is out of this world in comparison to something like Lambda.

It is ridiculously expensive. And not only that, what you provision, how many requests per second can it actually process? And he asked these very important questions. And I thought, you know, it was quite refreshing. Yes, it's incentivized to say that and to push you towards Lambda. But at the same time, he's actually right. There's a lot of control plane in there. And recently I've spent time over the last year creating the FASD project. Initially as an experiment, could we just take container D and container networking, the two lowest level components in Kubernetes, get rid of everything else and just run open FAS on that?

And the answer is, yes, you can. And you can then package it as an appliance, just like VMware did. But whilst VMware has Kubernetes, their own operating system, container networking, and then contour and all these other things, plus the connector. When it comes to FASD, you just have Linux, system D service, and one binary, that's it. And it's very efficient. I've seen like a real sort of interest from hobbyists through to commercial users. And there's a company called Sprucey in China using in production.

And so KubeCon Serverless Summit had a chance to give a 10 minute lightning talk on it. And just go over those questions that Adrian had and how FASD can sort of answer that. That's really cool. I've seen that in the insiders update and that's really, really cool. Speaking of the insiders update, let's wrap up the conversation with what if people want to support open FAS? I know last year, the CTO advisor, we supported open FAS by sponsoring the website. That was a pretty heavy commit from our part, but what are the ranges?

How do people get involved? You talked about the, this is your full-time gig. How do people support you? Is it like a Patreon? Like how do we support open source? I originally thought that that was what would happen when I launched a company sort of 18, 24 months ago. And I realized that that isn't the direction I want to go in. I'd rather offer a product that people can buy. Not interested in donations. If a company sponsors open FAS homepage, they're getting impressions.

That is a product. If a company wants to use open FAS in production and have somebody to speak to when things go wrong or be able to have a roadmap review every three to six months, they can buy an open FAS premium subscription. These are products that you can pay for through the company with an invoice. And then if you're an individual that, let's say really respects what I'm doing or values open FAS users at home or any of the other tools we make in the community, you can get this insider subscription, which gives you, again, a product.

It's not donation. You get a weekly newsletter on cloud native, all of these blog posts, some exclusive content as well. And that's just a great way to keep connected. And nowhere is anybody donating money. There's always an exchange of value. And it's always a net positive. And so if it's not of interest to you, don't do it. But I think you should try it, particularly if you have any interest in infrastructure and cloud. And what's the homepage for open FAS LTE?

com. Yeah, that's right. All right. Well, Alex, I appreciate you taking the time out of your schedule to come and talk to us about, first congratulating you on the best of VMware award, giving us some really great insight into how that fling was created inside of VMware and how it's matured and just how, you know, if the Kubernetes road or journey is right for you, how to determine that is very, very helpful. Insight. That's it for this episode of the CTO Advisor, CTO Dose crossover.

com. The next couple of weeks or so, we'll be launching a new homepage. We're pretty excited about that. Until then, you can talk to me online at CTO Advisor on the Twitter or on LinkedIn. Talk to you next CTO Advisor podcast.