Readout of Kubecon San Diego 2019 - With Alex Ellis of OpenFaaS

Machine-generated from the episode audio and not hand-corrected, so names and technical terms may be imperfect. The audio is authoritative.

Hey, how's it going? It's Keith Townsend from the CTO Advisor, and you're watching, I think this is the first time I've done a CTO Dose remotely, but I think it's a pretty good reason. I have on the other end, Alex Ellis, a good friend of mine's and community leader for Open FAS. Alex, how's it going? It's going great. S. All right. , you say that because you're, obviously, your accent, people might have thought it was from Texas, but it's not.

, so we appreciate you wrapping up. This is the end of your day, kind of the beginning of mine's. By the time you go for your sleep, this should be out actually. Great. We both crossed swords a little bit over at KubeCon in San Diego. We don't get a chance to see each other too often in person, and KubeCon, I think, was a really great excuse. Overall, what did you have going on at KubeCon this year? I had a very busy week.

It was really expensive to get out there, and I wanted to get maximum value, so it started off for me with Saturday at the Cloud Native Rejects Conference, and this is the second time it's run. It's a pre-event, and it's run by Chris Coole of Kinvulk, and the basic idea is if you didn't get your paper in, and by the way, I think 80% of papers submitted didn't get in, so there's a high chance of that, you get a chance to have an audience and get together with some folks that are coming a little bit early, talk to them about your idea, and it was a great atmosphere.

The other thing that that event does is it allows people that have got new stuff that basically came out before the CFP ended for the KubeCon, they can come too, and so that was my category, and I rocked up and I spoke about Inlets, which is a service tunnel and a reverse proxy that I think is going to be exciting when people start to realize the kind of things they can do with it. So briefly, I know you sent me a little info on Inlets.

Why do you need yet another reverse proxy? What's unique about Inlets? What problem does it solve? So it goes back to when I was still with VMware last year, and over the holidays, about a year ago now, we have this problem where the team needed to be testing webhooks to integrate with GitHub and other services, and there was just simply no way of getting them from the internet from a third-party service to their laptop where they were developing on Kubernetes locally. There was some existing tooling like ngrok that many people may know, but there's just absolutely no way you're getting that working in a corporate environment.

So what I wanted to do is just see if I could spend a week just messing about, see if I could create something that would work for that, and that became Inlets. Okay. I think that's the story of how many open-source projects, not just open-source projects, but companies are birthed. There was a problem that you needed to have solved unique to you, and you find out you're not the only one that had that problem. Yeah, and at the time, I was happy if it was just going to help our team get the job done, and then I put it onto Hacker News, and it got like two, two and a half thousand stars very quickly.

Gave it a couple of months or so, and spoke to Darren Shepard. He had some ideas of how we could refactor it and use some of the Kubernetes code inside of it. 0 and got up to almost 5,000 stars after that, and then people started messaging me about stuff they were doing with it. One guy copied a VM image from his NAS across the internet to another location securely, and he was able to do that in a few minutes. Other folks are looking at command and control at the edge.

I have a company that's working through my consultancy right now. They're a client, and they have different cinemas where they've got point-of-sale devices. They want some way to kind of manage them centrally, so the problem is that you can traditionally not get into those firewalls and networks. You may not even have any control over them, so if the client talks first, you can pretty much always initiate HTTP traffic. You can always get out, and then once you're out, you can have a full connection.

So this, I think, is a great gateway to talking about KubeCon in general. The whole thought of that, like my network admin, network engineer days, screams like, oh my god, this is horrible from a network security traditional sense where the network is the heart shell, and I'm the network infrastructure guy, and I control all of that. KubeCon was, and I think it's more apt to call it Cloud Native Conference, was just full of stuff like that. It was just project after project that the control of the infrastructure was pushed closer to the developer and the application.

Is that fair to say? That's a pretty good... I think that's pretty fair to say that, and you know, what I noticed at this KubeCon more than any other I've been to before, and I've been to a few now, is that there's so much content that there's now pre-conferences dedicated to just one topic. So I'm pretty sure it was the Service Mesh Con was the biggest that we had. It was also running observability, and I was a member of the Serverless Practitioner Summit and also helped to co-organize that.

So actually on the day there, we had a talk from the OpenFaaS community, and I gave a talk on what I'm calling the Plonk Stack, and the Plonk Stack is a number of Cloud Native projects, and the idea is to try and reframe serverless as something that Kubernetes operators and developers can make use of, because quite often I'll hear this term, and I don't think people really know what they mean when they say it, but they're like, we're implementing Kubernetes, we're not ready for serverless yet, and so wanting to try and reframe serverless and Kubernetes is really just an application stack that you can use to just move a bit quicker.

Yeah, it's odd that people say that, because I go into a show like KubeCon, and it's a bunch of building blocks, it's a bunch of bricks that you can put together to make something complex, and for those of you new to me and Alex's relationship, I sponsored the OpenFaaS homepage, not because I'm a big lover of Kubernetes, I look at Kubernetes as kind of a brick that I need to use to get to serverless. So serverless, I think, is the ultimate goal of many of the application architectures, and Alex's project, OpenFaaS, basically enables enterprises to have serverless functions.

So a lot of Kubernetes and KubeCon was about builders building different things, and for my little, I call it my little monkey enterprise brain, I have to have end products and projects that leverage this stuff. I'm not a builder in that traditional open source and cloud native way, so I need projects like OpenFaaS to kind of wrap my head around it. So with that said, what were some of the OpenFaaS related talks you gave at KubeCon? Yeah, so there was a new friend of mine from a company called Octato, and they were in the serverless summit and showed us how you can get this, what's being called in a loop development with OpenFaaS, and they can apply that to Kubernetes.

So the idea is as fast as you can type code, the Kubernetes cluster can reload it so that you can debug it and you can get this fast experience when you're building out your code. Now, traditionally, you're going to have like a lag of two to five to ten, fifteen seconds, depending on what it is you're building and reloading in the cluster. So from a workflow perspective, how does that, because as I think about typing code live, I'm not a coder, but you know, I do scripting and whatever I need to do for infrastructure stuff.

And as the type of code that I create, like I don't see the immediate benefit of applying that live, because most of, and this is, I think is a great difference between legacy approaching to application building and modern application building with serverless constructs. The type of coding that I do is normally batch processes. So I don't need to see a live representation of that code. How does that, give me an example of how does that help a modern application developer? Yeah, I mean, I think that's fair from that perspective.

However, when your batch job is failing in a way that you didn't expect, being able to get live into the code and put a break point and hit that line and see what the values are in the memory, that could be really invaluable, especially if it's like costly to run that operation. Okay. So say I'm making a call to a service that transfers data and that every time I make a call to that service, there's a cost associated with it. And that's where the code is breaking.

And every time I complete the batch process, I have to pay to have that test done. So as opposed to that, this would help me in that. Let's say you're actually moving money every time. Let's say you're doing payments on Stripe, but some of the time it fails. So you can attach a debugger and start to get some data out of that. It's not the only mechanism that exists. The other one is if let's say you're rendering a web page and you want to keep making incremental changes, shift the left a little bit, left a little bit more.

Like when we added the CTO Advisor logo to the homepage, you know, the workflow is add it, preview, move it a bit, preview, change the CSS, preview. And so for front-end developers, it can be pretty important to have that. Some of my friends at Garden are building similar products that gives you this inner loop and another company's tilt. And so there's three big companies I'm aware of, or big efforts, and they're really concentrating just on that developer experience. Whereas traditionally in OpenFaaS, what we've been focusing on, like with the Plunk stack, it's giving you some abstractions to work faster.

In fact, in the presentation I gave, there was like 10 or 12 files, 300 lines of text you needed to say, hello world on Kubernetes, on an HTTP server. In OpenFaaS, we could reduce the code to like eight lines. So it's interesting, you said, I watched one of the sessions from a chat from Google and he was basically saying the exact same argument. And he talked about how serverless speeds up development on Kubernetes. And OpenFaaS was one of the projects that he mentioned.

I tweeted it out. It's really interesting, especially you see coming from Google, who's kind of behind Knative. And for those of you who don't know the politics of OpenSource, Knative is a competing framework actually for serverless. There's a whole lot of politics around that, but we'll do that. It's a very similar effort. And I think you're speaking about Evan Anderson, who is a really bright guy. And I got a chance to meet him and the rest of the Knative team at KubeCon.

And interestingly enough, many of them have been hired over to VMware, where the two of us used to work. That's a story over drinks. Yes, I'm looking forward to seeing what will come of that, perhaps something with Pacific. Now, this event had 12,000 people apparently, and you and I both bumped into each other very randomly. And I think we were in the gangway and there was this huge throng and noise coming out of the expo area. I mean, how did you find just being in such a massive event?

So, you know, I've gone to reInvent, I've gone to VMworld, and I've gone to other events that are a little bit bigger. So the size didn't bother me. What was kind of off-putting or shocking was the number of sponsors there collecting lead leads. So it was kind of a it was a shock because this is my first open source conference. And my keynote experience was very, very different than what my traditional enterprise IT show keynote experiences were. But the show floor was exactly the type of show floor that I expect to see at an enterprise IT conference.

And the online Twitter chatter was kind of, it muddied the online Twitter channel because I expected, after seeing the show floor, I expected an enterprise conference. It wasn't an enterprise conference. It was an open source conference. And the two worlds kind of collided for me. What about you? Yeah. Well, I haven't been to VMworld or reInvent. I think, what are we talking, 20,000 to 70,000 people at those? Yeah. So VMworld at its height is 24,000. And then I think reInvent is expected to be somewhere between 50,000 and 60,000 people.

I just can't imagine how you'd operate in a conference of that size. I mean, the first one I went to, big one, was a DockerCon in 2016, the first time going to a tech event in the US. And I was pretty overwhelmed. And I think it was a couple thousand, 4,000 then. So to go three, four times that, I sort of hardened my shell, if you like. But I still find that I don't get to see what I want. And most of my time is spent chatting with people that I know or getting new connections in, we call it like the hallway track.

Right. Or dropping into, I do like to go to the keynotes. I'm not quite at the point where I can just sit in my pajamas and watch it in the hotel room. But I do want to be there and feel like I'm a part of it. Especially now as a CNCF ambassador, we get reserved front row seating, which is- Oh, it makes it easy. Yeah. So this was the first time as a CNCF ambassador. I know there's a whole bunch of us now.

And that's nice, having that group of people to hang around with and to know at an event. Some events I go to and I know nobody at all. So at least having that common core and the contacts I've built up has really helped me. But I could imagine somebody coming for their first time feeling a little bit like a fish out of water. So that was my expectation, but that wasn't the case. I ran into at least 50, 60 people that I know at the conference.

From the overlap between the Kubernetes community and vendors and customers who are coming to Kubernetes or KubeCon for the first time. What was really interesting was the number of people that had the same reaction as I did. This, again, is my first- I'm coming from the other end. This is my first open source conference. And it was just, it was surreal and it was strange. And I couldn't put into words. If I was in a traditional customer enterprise role and I came back and I had to give a conference report and say, hey, here's what we can benefit and use over the next six to 18 months, I would have struggled to come out with a clear message of what we need to do from an enterprise perspective to do it.

So as if I was a builder and I was building, you know, if I was building OpenFaaS, a Linkerd, SED, if I was in one of those projects, if I was building a cloud native company that was leveraging cloud native technologies to build a service, that was plenty of stuff. But for, you know, the regular enterprise consuming products instead of projects, it was a show that I struggled to see immediate value. Yeah, I think that's something that, you know, KubeCon is trying to address and CNCF through the end user talks.

At the last event, we had three or four end user talks with the OpenFaaS community in Europe. And I think that really helps people understand, like, what can you do that's real? And during the talk that I had with Linkerd, I showed a story from OpenFaaS limited customer that's doing cloud migration with us. And I showed an example from the community to try and give a bit of a balance between the commercial real life enterprise and the community open source side of things.

That talk was actually on the Wednesday. And I spoke with Boynt and a field engineer named Charles. And really, what we were looking to do is tell the story of how you can get a multi user platform that's secure, pretty much out of the box. I mean, there's always some things you can tune, but Ian Coldwater in the keynote, I think on Thursday mentioned how Kubernetes just isn't secure out of the box. Right. Whatever we can to kind of lock it down.

And then the idea is that an IT operator or a platform engineer could deploy OpenFaas cloud on maybe a managed Kubernetes service. That's what DX is doing. And then their engineers as they come, they only need to deploy their code into a GitHub repo. They don't actually need to really think about Docker images or Kubernetes. And a lot of that is offloaded. Right. So as close as we can get to the managed experience that you see in Google Cloud Run or maybe in AWS Lambda, you can recreate that for a team or for a company.

Yeah, I've. If anything, I walked away from KubeCon excited because there's a lot of effort between CNCF and I sat down and talked with Brian Lowes, who's the co-chair of KubeCon, or at least was the co-chair of KubeCon and the desire to get value on both sides of this community. One side of the community are the builders and the open source project participants. And then the other side of the is the consumers, the end user, the end customers who just simply want a product.

And then I think there's a squishy middle, which is a lot of people that I knew, which are enterprises that are giving back to open source. And that's probably the best model. Enterprises that's giving back to the open source community via cold contributions, donations, et cetera, and that are really involved. I think that's the model that I'm trying to take by even sponsoring the OpenFAST homepages, say that, you know what, I want to be involved in the process the way that I can.

And I think this is the model that I'm modeling for people that I advise. I think that's a great way to go about it. And if you know of some companies that might want to chat with us, do let me know. Something that comes to mind is, you know, there's a company called Weaveworks. And I know Alexis Richardson, the CEO, pretty well. He's a friend of mine. And he and his company built out this tool called Flux, which is for enterprises to use.

And it's similar to OpenFAST Cloud in that you put all of your code in one place and Flux will then apply it to your cluster. If you have a disaster and you lose your whole cluster, you bring a new one up, Flux will restore everything back to where it was. Now, over at Intuit, a traditional enterprise company, a bit like ADP that I used to work for, they're actually probably an early adopter and they built their own product just like Flux. And so this is what you're talking about.

You have this early adopting enterprise that built this open source project. You have this Kubernetes consultancy on the cutting edge. And the two have actually come together and merged the projects as Argo Flux. And the panel that spoke about GitOps was so popular that they had to move it to the main keynote room. Wow, that is. And I think that's a validation of that model that I've, even before Kubernetes and going back to OpenStack and pre-OpenStack, I kind of promoted this idea that enterprises have to start to develop their own chops and be able to contribute back to open source projects and use these projects in a way to differentiate their businesses.

I have questions and doubts of my own if most enterprises can scale that because of the shortest of the cross-domain talent that's needed to do that. But I think that's the best model. And I think that's the ideal model. So let's close out with this question focusing on OpenFaaS because we both have a vested interest in OpenFaaS being a successful project. If people wanted to, if people were curious about serverless and they were curious about having serverless being portable across cloud providers, I think OpenFaaS make, and projects like OpenFaaS make a good argument for adopting that as a solution.

Two questions I get consistently when I was at KubeCon, when I was talking to kind of my enterprise folks was one, how do I get started? And two, what, a little bit more technical, and I looked this up after the show, what are the triggers for the events or what are the, how do you trigger a code in OpenFast? So first, how does the average user get started? And then two, what is the experience versus Lambda? I know I can trigger off AWS events in Lambda.

How do I trigger functions in OpenFast? I think that makes a lot of sense. Something that I covered in a couple of the talks that I gave over the week as well, because it is a popular question and it does keep coming back around. But one of the things I want to do is trying to segue into that and say that what I'm finding is that people will do almost anything to avoid the full-on complexity of Kubernetes right now. Darren Sheppard over at Rancher Labs has been developing something called K3S or Kates.

The three stands for like halving the Kubernetes down the middle. And what that does is it reduces the memory footprint from something like two gigabytes for a master down to 300. And every, another called servers and every agent that you add in every node only takes about 50 megs of memory. And so you've got this Raspberry Pi cluster behind me and I can run a full scale HA application on it using K3S. So a lot of the work that I've done is around how can we make K3S really easy to use for developers, particularly so they can try OpenFaaS, whether that's on the Raspberry Pi or on the cloud.

And also how can we then go and get it easy to install all of the parts that you need for a useful OpenFaaS, right? We can do development, we can do production, and often there's a few steps involved in between. Helm just had a release of Helm 3 recently and that product is super popular, right? It's a CNCF project now. And what you can do is you say, I want a reverse proxy, I want TLS, I want OpenFaaS, and I want something else.

The unfortunate thing is, is that every project has this page like this on their docs, it's really hard to find. And so if you want all of those things, you're going to be spending a couple of hours scrolling through and adding them. As part of the work that I was doing on K3S, I created a tool called Ketchup. Ketchup can create a cluster and join the nodes, but the second thing that people don't really know about it yet is it can install a Helm chart with one command to any Kubernetes cluster.

And so if you want to get started, the easiest thing you could do is get Ketchup, get Kubernetes in your favorite place, whether that's EKS, Google, or like an Amazon EC2. And then when Ketchup app install OpenFaaS, and that will bring the Helm chart in and deploy it for you of authentication. You then maybe want TLS certificate with that. Well, you can then do app install cert manager, app install nginx. And finally, we can generate the ingress and the issuer for you.

Stuff that's normally like this much YAML each, we can do in one command with a strongly typed flag. And that means you just go app install OpenFaaS ingress dash dash my email dash dash my domain. That's fairly simple. What we did for Kubernetes, getting a deployment on Kubernetes, we've now done the same for getting all of the ecosystem tools that you need into OpenFaaS. So the second question that I get all the time was triggers. What are the triggers for functions and OpenFaaS?

So there's two models that open source serverless projects use for triggering stuff. One is you put the client subscription in your code. So if you're coming from Kafka, every piece of code you write will have this duplicated code, and it's not short. And then it will just start pulling in. Microsoft have a project called KDA. That's how they run. Knative and OpenFaaS both go the opposite way, where we deploy a broker that connects to the event source like Kafka.

And when it receives a message, it looks at what functions and endpoints you've got and triggers it off. And that's what we're doing now. So there's an SDK. And one of the end user companies, Form 3, built a SQS connector. We have an SNS example, a Kafka example, Cron, and a bunch of others like RabbitMQ. So really it's down to whatever is in the open source or whatever you want. If you can find some code to bind to your event source in a very short period of time, you can put something together that will fire off your functions for you.

And so basically the only thing that your function or OpenFaaS has to listen to is the broker itself. And just as long as I'm alerting the broker or the broker is, is it a push or a pull? Is the broker pulling? It's kind of a mix. The broker will pull in a list of all the endpoints deployed and look at an annotation called topic. Once it's read that annotation, it then goes, right, so if I get this topic, that function cares about it.

Okay. If I get this topic, these three functions care about it. And so then when it starts receiving messages, it pushes them out and your function doesn't have to even know that it came from Kafka or if it came from SQS, it will just get the message that it's expecting and it can process it. And then it just, and then whatever the function is, is, is whatever its role or microservice, it just does. Yeah. And then returns, whatever. So that's pretty straightforward.

I think there's a bunch of follow-on questions and discussions we can have around scale and et cetera. But if people want it to become part of the OpenFaaS community, do you guys have a Slack or how do you get, you know, how do you become part of the community? Yeah. com and you can find it in the docs. io. And that will give you a form to fill out and you'll get an invite within a day or two. All right, Alex, I really appreciate you taking time and letting your, your day over on the other side of the Atlantic.

For those who want to follow you on the, on the web, what's the best way to reach you? At OpenFaaS or at Alex Ellis UK. com and I'll be happy to chat to you. All right. And you can find me on the web. com. And if you have questions for me or Alex, you can, my DMs are open on Twitter. So talk to you next CTO Dose. Thanks, Keith. Thanks, Alex.