AWS Re:Invent Keynote Review - Blue Pill, Red Pill, Yellow Jellybean

Machine-generated from the episode audio and not hand-corrected, so names and technical terms may be imperfect. The audio is authoritative.

All right, while we were both at AWS re-event, we didn't find time to do this in person, but I have with me Enrico, so Enrico Signoretti, Senior Storage Analyst at GigaOM. Enrico, I've gone by years of calling you Enrique, so kudos to me to finally getting your name right. How's it going, my friend? It's going very well. Thank you for having me. It was a blast last week. We had a great time together, me, you, and all the other folks at the event.

Yeah, I tried to get Chris Evans on, but he's doing other things, but we ran into Quigley Pig on Twitter and a bunch of other people. It was an amazing event. 50,000 attendees, I'm told, so it felt like. 50,000 as far as I know, so it's a very good number compared also to many other events that I saw this year. They almost regained the same numbers that they had before the pandemic. It's very important because it gives you also an idea on where the companies are investing.

We were talking with some other analysts at the show, and some of their clients brought like 30, 50 people. That's amazing, right? Yeah. It's a huge investment, even if they are not paying for the pass. We are talking about moving a good chunk of your team at an event for one week. It was a full week. It went from Monday through Friday. There were sessions on Friday, so it was five days of content for folks who have never been to reInvent.

You kind of see the content online after the fact. There were sessions going up until 10 o'clock at night. They try and get as much content in as possible, so it's impossible to recap the entire show. We're going to focus on Wohner Vogel's keynote, which is day three or four. It's happening Thursday, the day after the big – actually, the party is Thursday night, so I can't even say the day after the party. It's the keynote that most of us, highly technical infrastructure folks, look forward to.

Enrique, for the first time in probably a while, he didn't really talk a lot of infrastructure. No. Actually, I mean, it was interesting because there is this huge push from AWS for serverless. I mean, while the storytelling was great, I mean, so he connected all the dots, started with traditional application development, why you should use serverless. The video at the beginning about comparing old, asynchronous ways of doing stuff with the new distributed, highly asynchronous microservices, et cetera, et cetera, way to do application was cool.

Maybe he spent too much on this, but I know that this is the push, and then he went on a path about the next steps, and he ended with quantum computing, so very, very long, let's say, path, very, very far horizon that people can – you can also target quantum computing in 10, 20 years, but it was a good keynote. Then we can debate on the value of serverless on this kind of microservices that maybe are too radical for most of the developers, but again, I really enjoyed it.

I did enjoy it, and I always enjoy his keynotes. He has an interesting style of presenting. He had a kind of a red pill, blue pill, yellow jelly bean deal before, you know, play on the matrix, so it was a fun – you know, Werner's type of guy seems like doesn't take himself too seriously, and he can poke fun, and it was a fun session, so I kind of like to divide it into like those three different sections. The first hour was kind of building the case for serverless.

The next one was kind of more of your wheelhouse data, AI, and analysis, and it kind of bled into a third area, which was, you know, kind of the future, and, you know, he went into kind of the power of now and what you can do with simulation, and then what you can do with simulation in the future, so it was – you know, we can kind of break it into three sessions. It was kind of two first hour on serverless, second hour on AI, ML, but with some, you know, subtopics within that AI, ML story, so what did you think about that first hour of just – you know, he – it seemed like a really good advertisement for Lambda.

Yeah, I mean, so in general, I love serverless with – but I have to be honest. When I think about it, so it speeds up development. It makes application really scalable. You don't have to think about anything. I mean, everything infrastructure is a problem of the service provider, and you just – you just make these components smarter. You connect them. You know, this is the real microservices world. On the other hand, what I find frightening, I mean, is that serverless is a huge look-in.

Okay, so when you start developing with Lambda and you start attaching this function to all these components in the AWS ecosystem, it means that in the end, you have an application that is totally AWS focused. So, you know, if you think about it, it's – it becomes really, really complicated then to think about different clouds, and when I talk with my clients, most of them have two, some of them three or more clouds, okay, and it's not because strategy in many cases is a technical choice or it's because one of their partners forced them to work with a different cloud for specific reasons.

Maybe there is a very specific AI ML kind of library that this third party used and, you know, and if you want that solution, you have to use that cloud or, you know, maybe it's a data governance problem or whatever. Anyways, so how do you port your application to a different cloud? I mean, it's quite impossible. I mean, you can rebuild the logic. That's probably the easiest part, but everything has to be rewritten to work in that cloud. Different APIs, different services, even the response times.

I mean, so because they are optimized in a different way, so different service work a little bit different. So something that you choose at the very beginning on AWS, because it works in a specific way, then maybe you will find a similar service on AWS, but it doesn't work exactly the same. So it will create a lot of discrepancies and inconsistencies in the way the application is developed. So keeping, you know, up with a single code base is a problem, you know, keeping up with multiple code base and maintain the same set of functionality could be a huge problem.

You know, we saw this multiple times in the past with iOS and and Android, right? So if you when you were developing the same application with, you know, native APIs and native development tools of the two platform, you had to rewrite most of it and the application were not the same. They were not aligned. Now many of the loopers use this, you know, ideas that are, you know, common for the platform that simplify development a lot because you develop the single application and then the system helps you to to maintain the application on different platforms.

So it's so you use a single language, you use a, I mean, a single framework in the end. So it's totally different, especially, you know, if if there is some complexity in this application. So that's that's a thing, I mean, yeah, and the lock in part of it is, you know, back when I started following services, maybe six, seven years ago, I had this this I struggled with the basic concept of event driven compute, because back in our monolith world, you know, we write an application and there will be a listening dame and there will be something constantly running, waiting to.

Kick off a function or in this case, an event that that will kick off a function to listen to. And if you're not writing that yourself, then what what became the thing that I struggled with was what was that constant running damon and learned did a really great job of breaking it down and say basically AWS is that damon AWS is that listening service, that bus, that message bus. You did a really great job, even whether you believe in in serverless or Lambda yourself.

It was a really great perspective of how the right distributed applications, whether you recreate the AWS functionality yourself, like within containers or different cloud services, great architecture discussion. But there's lock in. So if the AWS message bus basically is my core service and I have to write logic around that, it allows me to develop serverless applications really fast, save a ton of money, not kind of worry about databases. They did a lot of the keynote. The main keynote talked about, you know, ETL between a war and their other big database and a lot of great advantages.

But to your point, if I call to that now, I'm locked in to AWS. There's not equivalent, not one to one services. I can do a lot of the same logic in the other platforms, but not in the same way. So now I have two development teams with two different tracks and trying and then two operations teams trying to keep up with performance and SLOs, SLAs, etc. And it becomes kind of this big mess that I tried to avoid when I was building monolithic applications in my private data center or pre cloud.

So I really, really appreciated the logic around building distributed, modern distributed apps and how the overhead is worth that effort. But, you know, there's a really interesting tradeoff that customers have to consider. Do I do this with kind of an abstracted container type platform and use native cloud services sparingly so I can have portability or do I develop for speed and convenience with these native cloud platforms? Well, there is the, you know, the jelly bean way. I mean, so it's and maybe it was not explored enough.

I mean, I would be the guy that chooses the jelly bean, just not just for the flavor. But the idea was banana flavor. Right. So the idea is that you can build on top of every cloud today, you know, your infrastructure and make it portable. This is something that every cloud provider wants to avoid. But actually, if you look at, for example, my work, like Pure Storage or NetApp, they are, you know, through acquisitions or because they built it in with their teams.

So both Portworx with Portworx databases, services, PDS, right. And and NetApp with IntraCluster, they are building this mechanism to, you know, create a service that is, you know, built out of open source. So you create your databases, you create your framework, and they are always the same. You can port them no matter, you know, on top of Kubernetes, no matter where you are deploying your infrastructure. So if I think that there is OpenFuzz and there is Knative, you know, similar frameworks to do serverless, I mean, they can't be as advanced as AWS, because in AWS, you have everything integrated and etc.

But actually, if you're able to build your environment, you can use this, you know, development methodology. It works. And then you can move your application with your infrastructure on top of every cloud. I don't want I don't want to call it MetaCloud, SuperCloud or whatever cloud like others do. But actually, this is what it is. I mean, so the cloud is a commodity infrastructure, just infrastructure, not managed services. And then on top of it, you deploy your real infrastructure, so your PaaS.

OK, and so let's talk about that, because that's the I think if we broke the conversation down to like a carrot and a stick, the first part of the conversation was the carrot. Like here's here's what you should do and here's the advantages of it. And the second part of the conversation is kind of that stick like, yeah, you can build it with OpenFuzz or Knative or some other platform that's portable, but let's flex. Let's talk about this is something that we haven't talked about yet in the actual podcast.

Some of these customer stories from Siemens to startups around natural gas and et cetera, and this kind of power of the AWS platform. Yeah, I mean, so when you are at these events and you talk with these people, you really understand, you know, that I think that you have two options. One is full on, I mean, and you take advantage 100 percent of the system, the ecosystem that AWS is, you know, has been building since 2006. And they are the most advanced in this.

They provide a lot of services. So if you have a good architect, cloud architect that knows very well, you know, all the ecosystem and you have access to professional services from AWS. And of course, everything becomes easier and it's a good partner. OK, maybe maybe I have to say that, you know, others can provide you similar, similar, you know, solutions. But actually AWS is just bigger, more advanced, a little bit more everything. OK, so it has a true advantage. But again, you have to commit everything on it.

So are you ready to do it? Is it is it something that you really want to do? Because, you know, let's think about, I don't know, the basic things that are in the mouth of everybody in these days, like cloud cost optimization, OK? If if you are on a on a single cloud, OK, I mean, it's harder to do cloud cost optimization because maybe there are resources from other cloud for type of workloads that you do that, you know, are cheaper for just that specific workload or because maybe their APIs are more optimized or maybe because they do things just different in a different way that are, you know, more aligned with your applications, I don't know.

So I mean, it's a it's a very complicated question to solve, to to to give an answer. The perfect answer is impossible. When you see this, these guys with this very fancy use cases, you see, wow, for this specific use case, AWS did a great job. I mean, they gave you everything. So. Oh, and so this is where I want to end the conversation, because this is where it's practical, because we saw some amazing use cases from Unity to Siemens to, you know, simulations, city size simulations within EC2 and then AWS making services from that data, those data sets, et cetera.

Then there's the rest of us like there's us who struggle with, you know, just understanding all the features of S3 when a company says that their storage is S3, their object storage is S3 compatible. They're usually just talking about read, write, delete type of compatibility. But when you go into the AWS S3 panel and services, it's practically a database now. Like you can do some really crazy stuff with S3. The question is, at what level of extraction do you do you?

Make the decision that I'm either going all in in AWS and using these unique services around something like a S3 that only AWS provides versus, you know what, I'm going to use S3 in the read, write, delete narrative versus S3 as in it's a database and I'm going to look towards like a proper database outside of S3 so that I can have portability. Do you remember when, you know, in the past we were designing systems, complex systems, maybe an SAP system? OK, so you are thinking about SAP, but you are designing everything end to end, including the hardware.

And that was, you know, at the end of the day. So you were using a big Unix machine or or, you know, a different type of system. But actually you are designing everything and you were, you know, optimizing everything for that platform. I think the same is today. I mean, you can think about this. I mean, a startup or or Siemens or that we saw or, you know, everybody else. I mean, they have an application. They're thinking, OK, what is the best technology and what is the best stack end to end to develop this application?

And then it comes to AWS gives us this, this, this and this layer. They are really optimized. We like it, including S3, Lambda and other things. And why not build this application this way? Yes, of course, you are building a silo, but it's a very efficient silo. And so we started thinking about building generic stacks when virtualization started to appear. OK, so we started thinking about just the operating system as the, you know, lower layer for your application, because you were choosing between a Linux or a Windows system at that point.

But again, the same goes here. I mean, you can choose, do I want to be abstracted? And at the very beginning, virtualization was not as efficient as today, for example. So and so do I want to be abstracted and lose some of the efficiency or do I want to be, you know, bare metal and go end to end? So this is even if it's the term is used differently. This is bare metal cloud versus abstracted cloud. And the bare metal cloud means in this case, I mean, I know that usually we are talking about it differently.

But if for this example works, I mean, so I want to take all advantage of the metal and I want to to do everything I can with it. This is why I'm choosing this architecture. This is why we were choosing big SMP Unix machines in the in the 90s instead of Windows NT or whatever, the first version of Linux, because it was much more performant, it was more scalable, it was much more, you know, code where was optimized for it. Oracle was already doing better things on this hardware instead of Microsoft SQL or instead of Oracle on on Windows NT.

So there was a stack and it was working the same as with the cloud. You know, nothing changed in the end. You can have a great silo that works perfectly and perfectly optimized because this is your goal. Or you can start thinking differently, which will pay off probably in years. Which is not something that is feasible for many enterprises. So, Enrique, I'll leave the audience with that thought is it's. You can't ask every question when you're building an app or creating a new infrastructure, but you can start with some questions and those questions basically need to be, who am I building this for?

Why am I building it and what are the advantages of one platform or another? Basically, is the platform or architecture that I'm looking to leverage, is it meeting the use case? And a lot of times where me and you meet customers and we end up talking to customers and helping them with their problems is that they didn't ask these basic questions like they didn't, you know, they kind of jumped in and said, oh, AWS allows me these tools and they didn't realize that a my business partner doesn't want to use this cloud provider.

It doesn't want to give their have their data on this cloud provider. And now they end up in a really bad situation that they've made this major investment where they're sending 30 of their staff to AWS reInvent and realizing, man, I might have to send 30 of the same staff to Microsoft, whatever the Microsoft conference is. That's how big AWS reInvent is. Microsoft Insight, I believe, is. Yeah. Yeah. They have three or four conferences now. They have a lot.

And maybe this is a question that you didn't ask is what people do I have to, you know, pull this off? So, I mean, when you start thinking that end to end stack, you need infrastructure and developers. But think about serverless. You just need developers. Yeah, and it's a it's a really difficult question for a lot of startups, do I invest in recreating what's basically non-differentiate infrastructure? So some of these services that AWS already has for me. Yeah, there's a lot of open source out there, but I have to stitch those things together.

Even if I get a head start with something like OpenShift or Tanzu application platform, I still have to build a bunch of infrastructure to get my app up and running. Or do I spend that three hundred thousand dollars on developers who can just write more business logic code and I get my app to market faster than my competitors? So while and I have to just sacrifice portability and lose some business opportunities. These are not questions that are asked in the vacuum or answered in a vacuum.

For folks that want to find out more of your views on Rico, where can they find it and what's your latest project? Well, it's easy. I mean, you can find me by Googling my name. It's I am the only analyst in the IT industry that so it's not a really common name. com probably where you can find all my write ups and most of my content. And I also host a Friday live show on LinkedIn. 30pm Central European time.

30am Pacific. So just look for me on LinkedIn or or on Google and you will find me. All right. If you want to find out more about the CTO advisor, you can follow us on the web, the CTO advisor dot com. DMs are open at CTO advisor for me. Talk to you next CTO advisor podcast.