Replay-Serverless is bigger than Cloud – NY Times CTO 2018

Machine-generated from the episode audio and not hand-corrected, so names and technical terms may be imperfect. The audio is authoritative.

Hey, you're listening to episode, I don't know, Mark, it's been a long time since we've done a podcast. Yes. It's, you know, the new year, the holiday. I'm just getting lazy. So apologies that it's been a couple of weeks. You know, Mark was on vacation. I was on vacation. And we didn't let you guys know that it will be a while, at least two or three weeks before the next podcast. But that's OK, because, Mark, what do we have on deck for today?

So we have Nick Rockwell and we're going to talk about serverless and containers. Nick, why don't you introduce yourself? Hi, my name is Nick Rockwell. I'm the CTO at The New York Times. And we've been experimenting pretty heavily with serverless. And I've become fairly convinced that it's the way to go. And I'm sure we'll get into a little more detail about what we mean by serverless. But I'm sort of betting the farm and committing our whole team to drive ahead into serverless as aggressively as we can.

So I'm obviously pretty bullish and excited about it. So, Keith, I remember you wrote a post about serverless is more than AWS's serverless, which I would call function as a service or event-driven computing. So I think it'd be good if we can start with understanding what we mean when we say serverless now versus what we meant when we may have said serverless a year ago. Yeah, Nick, just some background. I got and I got beat up for this about a year ago.

Real remark is that, you know what, the concept is that developers just want to write code and push that code to some service that runs the code. To me, that's serverless. So there's a bunch of different things that may fall in that category. Just as long as my operation team doesn't have to worry about the servers. I don't care if it's functions as a service in the form of Lambda, Google Cloud functions or anything. You know, those event driven components that people called serverless a year ago versus, you know, we've gotten to this debate.

Me and you, me and you directly that pivotal, pivotal cloud foundry or cloud foundry in general run by someone like IBM off prem in a public cloud is, in fact, serverless. So, Nick, what's your view? Define serverless for us. My view is pretty straightforward, but I think the key thing is that there shouldn't be any operating system and there shouldn't be any instances of any kind. You shouldn't have to think about scaling. You shouldn't have to think about cross region reliability and the architecture at that level.

If you find that you're having to deal with any of those things, it's not really serverless. I do think that the, you know, Lambda and the various function as a service platforms out there absolutely count. But I also think that we have a tendency to focus too much on function as a service. It's not quite true that developers just want to write code. They want to build applications. And something like function as a service is probably the easiest way to write some code and get it running somewhere.

But it's not a great paradigm for building complex apps. So, I feel like overemphasizing Lambda or other function as a service platforms has been one way that people have dismissed serverless. There are other ways to do it. The trend is in a different direction, I believe. And that really makes it actually much more impactful. So, we're just now starting to get this. A lot of people are just now starting to get into containers, right? Countless people I talk to, oh, yeah, we're getting ready to deploy Docker or Kubernetes or whatever it is.

So, how is it that you're ready to bet the farm, so to speak, on serverless when most people are still hung up on containers? So, I look at containers in the classic sense, and I don't see too much to be excited about. To me, it doesn't meet the criteria that I just laid out. You're still dealing with an operating system image. Although it's simplified, it's still the same thing. And you're still dealing with scaling and capacity management at the cluster level.

So, I feel like people have gravitated to containers because it's pretty familiar and it's pretty comfortable. And it doesn't actually require you to re-architect or rethink too much, and it's sort of easy. But I don't see a whole lot of advantage in it. So, what is starting to emerge that is pretty interesting is fully managed container services, where the underlying cluster is completely managed by the platform. And this is kind of what App Engine Flex ends up being and what Amazon's Fargate looks like.

So, that changes everything. And then if the container is just the abstraction for deployment of the application, but everything else is fully managed by the platform, that starts to look like a pretty good serverless paradigm. So, if I'm not thinking about my container, if I'm just more thinking about my code and not drawing out on the whiteboard, here's container A, here's container B, it doesn't matter if they're – well, you think they'll be part of a serverless architecture, just managed better. Is that kind of what I'm hearing?

Yeah, once I don't have to think about the cluster that my containers are running on, and I don't have to think about the instances that make up that cluster and what those individual CPUs and blobs of memory are doing, then it starts to be much closer to being serverless. If that cluster is also managed across availability zones and regions, and I therefore don't have to think about availability and reliability either, it gets even closer. And then the less that I use a specific custom configuration of the operating system to make sure that my app runs and really just use the container to manage code dependencies, it gets very, very close indeed.

That's really what App Engine Flex ends up looking like. So, that's the path I think we're evolving down, but it's a big, big difference. So, Nick, I think we probably need to pause a little bit here and help people wrap their heads around this, because I'm sure there's a lot of folks scratching their head. Lambda was tough enough to get, and they eventually got it. Okay, there's some code that I can run that can kick off an event, but I still have this static.

At some point, I have a server that's either running some type of steady state application, and whether it's a storage array or something that kicks off an event, there's something that I can anchor my code and my thoughts to, even in the functions as a service model for serverless. What is that abstraction logically when we think about this building an entire complex app? What's that? Is there anything that we can anchor kind of our mind to, to grasp? We're no longer thinking about OSs.

We're no longer thinking about containers. We're thinking about what? I think it's the app, and that's very, very clear in something like Google's App Engine Classic, where you're deploying code, and it's a whole application, and it's managed and defined in a very specific way. The good news is that we need to build applications that are more complicated than a few functions strung together on a function-as-a-service platform, but also serverless helps to orient us towards making them nonetheless pretty simple. The marriage of serverless and microservices is very, very natural.

We find ourselves deploying smaller apps, and they don't have to be super complicated, and they shouldn't have a lot of gnarly dependencies. So we really just get to think about the abstraction of the application, and hopefully in a pretty simple way. I don't need to think about, okay, what version of whatever OS I need to run in my container. People with container infrastructures are still dealing with Meltdown and Spectre. This completely abstracts that away. I'm thinking about, okay, here's the business problem.

Here's the logic that I need to go through to solve that. Now let me go about coding it. That's exactly right, and the other thing that's kind of great, even though it's a little uncomfortable for engineers, is I also don't have – there's a whole bunch of decisions that I no longer have to make that actually I can't really make. So many decisions about how that app is architected and run become part of the platform that the process becomes much simpler for me.

Typically, if you're building an application from scratch, you start by making a choice about what database you're going to use and what language and what platform and what OS and how is it configured and what other services. Most of that goes away. So, again, like the path from here's a business problem to here's the code I'm producing to solve it becomes much, much shorter. So let's talk about your experience actually doing this. I know a lot of times we talk about things that we can do, we might do, we should do, but have you actually deployed much serverless apps in your environment?

Yeah, we have. My favorite one to talk about is our Crosswords app because it's tons of fun and also has some interesting challenges associated with it. But we've been focused on Google App Engine as our primary target for serverless, and we built out the New York Times Crosswords app on App Engine. And I'd say the hardest part was sort of training or getting engineers over the hurdle, almost of like how little they had to do, and how much you kind of have to just go with the flow of how that platform is structured and how it kind of goes about solving different kinds of problems.

And then once we sort of got through that, the feedback I got from the developers was, wow, this is great, this is easy, and I don't have to think about scaling it. And, you know, that was pretty amazing. So, yeah, I mean, our experience has been very, very good. What's a little bit, my next step is really to take, we also have invested a lot of work in deploying apps onto container services, and I'm really excited to start experimenting with deploying those onto App Engine Flex and Fargate to see how that feels because I think that's a great migration path.

So that'll help us move a lot faster. So, Keith, you know what this reminds me of is a conversation that we had several times, actually, where IT needs to stop worrying about IT and focus more on business needs. I think it's interesting that we've already kind of started talking about that with infrastructure especially, and now I think that conversation is naturally moving towards a development. Well, I love first off, Nick, that you guys were brave enough to try this with the Crosswords app.

Our good friend in the community, Sarah Veller, is a huge New York Times Crossword fan, and if there's problems with that app, geez, oh, I couldn't imagine that, heard times a few million people. Oh, yeah. So, you know what, Mark, and I think you're right, and this is, you know what, we're proponents of this whole concept that IT needs to get out of the business of turning knobs and, you know, managing equipment, but I just, just personally, I feel some angst that, what do you mean, there's no knobs in serverless?

You mean I can't even set the amount of RAM that I want in an instance? What if I need more? Yeah, what if I need more? And I think that's a good thing. I think the fact that even I'm feeling a little angst about this is interesting. So, Nick, I'd like to ask you about that kind of cultural change that you guys have had to go through. What has that been like? That is the hardest part. You know, we ask, we all, like, work so hard to hire great engineers, and now we're kind of asking them to not worry about a whole bunch of stuff and trust the platform provider to take care of it for them, and that's hard.

The best engineers, typically, when you're trying something new, you put your best engineers on it, and they're the ones, frankly, who love poking holes in things the most. They're going to spend a lot of their time trying to find out what's wrong with the platform or what doesn't work or what they can't do. And I think you've got to let them have a little time to do that. Very often, a lot of this stuff is pretty raw, so they'll find some things, and you learn some valuable lessons.

But hopefully the next step is you've got to get them to the point where they actually get excited about that, realize how much more productive they can be, and then, like, give them plenty of, like, work to do with its actual application. So, Mark, we've been through that. You know, it goes even when something as simple as virtualization. When virtualization first came around and, you know, engineers first started to think, wait, we're going to go from having 100 servers to 25 physical servers?

What job am I going to do? And we soon discovered that IT, the IT quickly gets complexity, complex as it scales. As you give, you know, this is that Javon paradox. As you make something more easy to consume, you're going to get more consumption. And one thing we know in computer science is scale breaks stuff. Yeah, I definitely agree. As things get easier, things are going to get more complicated. But, you know, when your platform is built to help you reduce that complexity, hopefully it frees up your time in other areas that you can actually focus on providing business value, which is, I think, what we all need to be doing as technology leaders.

Nick, what are some of the pointers you'd like to either culturally, technically, or even process-wise that you learned that you're like, you know what, this is where the industry itself is short at, or this is where you really need to get on board with the concept right away? I mean, I'll say, as I open with, like, to me, this is a big one. Like, serverless is a big deal. I think it's bigger than, like, the first wave of cloud. It changes more and has the potential to have a larger impact.

So I'd say, like, pay attention to this one. I also think for management types like myself, this is one where you have to get involved and go pretty deep. Because, again, you're asking your engineers to change a lot of habits and learn a lot of new tricks, and they're just going to need to see you, like, leading from the front and, like, getting involved and pushing them a little and listening to them and helping the whole kind of organization and team, like, find their way into this new way of working.

So that's what I would say. Like, this is one where it's worth going deep, and, you know, the payoff's going to be there. So strong encouragement to do so. So you've talked about this internally at The New York Times, but also you've written about it and given a few presentations. What is the feedback that you've heard from audience members and other people when you start talking about this? It's so jarring that I can't imagine a lot of people just instantly get it.

You know, I've actually been surprised. When I first started talking about it, I really thought, am I missing something? Because it seemed so clear to me, but I sort of felt so little enthusiasm from, you know, most of the people I encountered, there's so much focus on containers. But as I've talked about it, the overwhelming response has been people coming up to me and saying, like, yeah, of course. Oh, my God. Like, I hadn't really thought of it quite like that.

But it's been quite easy to convince people, and I've really been, like, trying to find people to convince me that I'm missing something or to think about it differently, and that hasn't really happened. So I think it's just like we're at a moment where I think the momentum is actually picking up very quickly, but I'm finding that people do get it and they jump on board. They just kind of need someone to say it's okay. Yeah, I would love to talk to you in a year from now.

And I have this theory that stuff like serverless is going to make enterprise IT, the traditional sense, shrink. And at some point, we're going to push this logic so close to the business user that IT itself is going to be like the definition of what an engineer in an enterprise IT organization is going to be is going to completely change. I wouldn't say it's changed. I think it's going back to what it used to be. There was a time when IT was a function of the business, right?

We were co-located as part of the business. We've moved away from that. So I think as we go back to that, I don't think that's a bad thing. I agree. I like to do cooler stuff than fix boxes. Me too. So I think this is really great. I've all said for a long time that containers are just a stopgap as we move down the ecosystem. It's just a smaller iteration of a server with some orchestration around that, and so I'm glad people are starting to get that.

So, Nick, any final thoughts? No, I'd encourage anyone who's listening to, again, this is a good one. This is going to be big. It's worth digging in and rolling up your sleeves and understanding this and being a little bit brave. And then also talking to the platforms. I feel like they're a little in a place where they're not quite sure what we as IT consumers want, and we need to tell them. We need to tell them to keep investing in their serverless platform and keep pushing, and then we need to tell them to pass a lot of the savings back to us.

So I encourage everybody to get involved, learn about it, start some stuff in your organization, and then talk to the platforms and let them know what you're doing. Cool. So, Nick, where can people find you online if they want to follow and connect with you? They can find me on Twitter as NickSRockwell, and on Medium it's the same. Awesome. And I am, of course, Mark May of Syncy Storage. Keith, who are you? I'm at CTOVisor on Twitter. Awesome.

And, guys, don't forget, go to your mom's house, and when you're there, sneak into her iPod, her computer, all her devices, and just go to her podcatcher and hit subscribe. Thanks, guys.