Serverless & IBM Cloud - CTO033

Machine-generated from the episode audio and not hand-corrected, so names and technical terms may be imperfect. The audio is authoritative.

Hi, you're listening to episode 33 of the CTO Advisor Chat. So I took a week off. Hey, you know what, it's a part-time job. Really like this conversation. I'm joined by Tyler Britton of IBM's Blue Box. We have a great conversation around exactly what is Blue Box? I know IBM made the purchase. I know they're open stack play, but what exactly does it do? Where's the value add? Tyler addresses that out the bat. And then we get into a deep conversation around serverless.

I had the YouTube video doing a high level explanation of Lambda. Tyler's actually a lot smarter about this stuff than I am. So I get to ask the expert questions and really dig into this serverless compute thing. How does that look on a backend when you talk about an open stack or IBM's Blue Mix? Great conversation. Hope you guys enjoy it. So Tyler, go ahead and introduce yourself. Yeah, thanks, Keith. My name's Tyler Britton. I work at Blue Box, an IBM company in the office of the CTO as a technical advocate.

So focused on getting out there and talking about Blue Box, what we do, and all those good things. So speaking about Blue Box and what you do, I've heard the brand on Twitter. The Blue Box Jesse went at the meetup the other day. I think that was Jesse. I didn't realize that was Blue Box Jesse. Went out of my way to talk to him. But the name, I think, precludes or goes before what you guys do. What do you guys actually do?

So Blue Box is, in the simplest terms, open stack powered private clouds as a service. We deliver them in customer data centers or in this case IBM software data center. So any place IBM software is a data center, you can have your own open stack private cloud on hardware in your data center. So me and you get into conversations on Twitter all the time about the value add of infrastructure and its place in the business. We won't get into an argument.

At least, I don't expect to get into an argument in a podcast about that. But what's the value add of a Blue Box as you see it from your lens? That's a fantastic question. I think one of those things that we almost strenuously agree on is infrastructure by itself is of little value. It's a supporting piece for something that actually provides customer value and business value. So from my perspective, what Blue Box gives customers is a repeatable, dependable infrastructure as a service built on open APIs, open technology.

It allows you to build additional things on top that actually do provide value. So those could be something that you bring as a customer. You use Ansible to deploy your special app or whatever, and it can be deployed on VMs on Blue Box or even within IBM. So if you want to use Cloud Foundry or anything from the Bluemix catalog, we already announced this year at Interop that we're going to be able to supply all the Bluemix services on Blue Box as well, and things like Watson.

So from an IBM perspective, IBM will be able to deliver more value add technologies and solutions on top of a dependable, turnkey infrastructure as a service layer. So I like that idea. Basically, we've all heard the stories that OpenStack is hard. OpenStack is hard, but there's every vendor out there from a API perspective supports it. So if you want to build a private cloud, a hybrid cloud, your options are pretty limited. You'd have to go with an OpenStack type of solution, and Blue Box basically makes that easy.

It's all about simplicity and portability, right? So you get those OpenStack APIs that you can take anywhere. We don't do any sort of magic secret sauce to the OpenStack code. It's the community OpenStack code. The real value we're bringing is that operational engineering expertise and excellence. So not just do you get an OpenStack cloud when you turn it on, but all upgrades are included, all troubleshooting, all the pieces. You get, as a customer, Jesse loves to say, you get to work with OpenStack, not on OpenStack.

So you get to- I like that. You get to do the thing that actually adds value, and you're not spending time screwing around with the infrastructure itself. One of the things that's really interesting to me is this migration or shift from where we're focused on the APIs and how we're going to deliver the APIs. Is it going to be OpenStack? Is it going to be AWS? Or is it going to be vCloud and vCloud Air, which is another conversation for another day.

We're starting to even get away from that concept. You invited me and thank you to a meetup last week or week before the weeks are starting to run on on the concept of serverless. This is something that I had heard on a CloudCast podcast, but I still, I couldn't wrap my head around it. And this meetup was perfect for explaining to me exactly what serverless is. Can you take a crack at it? What's serverless? And I would love to hear kind of your roadmap from a OpenStack and API-driven environment to a serverless environment.

Sure. I mean, I don't love, like most things when it comes to branding, I don't love the name serverless just because I think serverless has existed before this. The idea is you as a developer, programmer, company, you don't care about server one, server two, server three, it's just running code. So to me, things like Cloud Foundry and OpenShift and solutions like that are already serverless because you're just giving a code. You have no concept of servers. What we're talking about now, when you see solutions and technologies talking about serverless, what they're really talking about is that's something I would prefer to call event-driven computing.

So event-driven computing is all about if my code's not doing anything, why should it be running, right? Why can't I just call that code or function whenever an event happens, execute the code, do whatever it needs to do, and then turn that code back off until the next time the event happens? So from a development or programming perspective, if you're describing a part of your application or code and you say, well, it waits for X and then it does Y, that it's a perfect candidate for this type of solution.

So event-driven services or event-driven functionality, how is that different from platform as a service or is it the same thing? I think a platform could provide that. They don't do that by default. But I think the biggest difference is right now, platform as a service, say something like Cloud Foundry or Beanstalk or Heroku, I give it code and it fires up that code and runs it, whether they call it an instance or a dyno or an app instance or whatever, that code's sitting there chugging along, running in memory, chewing up resources, just sitting there idling.

So like if I had a piece of code that did something really basic, right? So let's say I was building an app that if you uploaded a picture to my website, I would apply some magic filters to it and clean it up and make it look nice and resize it to make it ready to put as your profile picture for Twitter. Well, if my app was running on something like Heroku or Cloud Foundry, that instance would be up and running, basically not doing anything unless you uploaded an image.

The difference with something like this, event-driven, is that code could be completely turned off. And when that first call comes and you upload your picture or call the API, the system that's operating it will wake up the code, feed it the input. It'll, you know, say in this case, it'll process the images and then return you with the output, the new image for your profile pic and then turn back off. And from a operational perspective, it's nice, you know, you're consuming less resources, but it's really almost even more about cost and consumption.

So if I'm paying, say, Heroku, you know, whatever dollars per month per dyno to run it, my app, there's a cost associated with it, with serverless or event-driven, if that app's not running, I'm not paying anything. I only pay when it runs, and then also gives me infinite scale, depending on the platform I'm running it on. So if someone, you know, if just so happens, someone talked about this photo app on TV, so suddenly all these people are downloading and starting to use it, and I get, you know, 100,000 simultaneous inquiries, the platform will be able to spin up this event-driven code, you know, for each individual instance, so they won't notice any difference whether one person requests or 100,000 people request it.

So I like that, one, the scale piece, that's a very different model. And this is different than we have even in the cloud. So in traditional infrastructure as a service, if I'm consuming a OpenStack or a AWS infrastructure, I have to have some type of controller node within my application that scales the app up and down via APIs to the AWS or OpenStack control panel, basically. So I'm telling Nova, you know what I need? 10 more instances, the demand is gone, those scale back down to two instances, is basically the difference in the platforms from a scale perspective?

Yeah, from that perspective, I mean, they're both OpenStack with heat autoscaling and Amazon with autoscaling groups, you can kind of, they've put a little bit more around it now where you can say like, hey, these are my app instances, and if you see this stuff, make more of them, once it calms down, make less of them. But there's still, there's always at least one instance of that app running at any one point in time, because it has to be to service those users where this code is basically called, executed, gives you an output and then goes away, and there's a lot of different uses for it.

So, you know, the first kind of, the first entry onto the scene of this type of technology is Amazon's AWS's Lambda. Not only can you call Lambda from a lot of different things in AWS, like on a schedule or on an event, like if someone uploads something to an S3 bucket, but you can also use it with their API gateway to basically build an API using just a bunch of these functions. So in turn, this is basically a supporting technology for a microservices-based application.

Yeah, yeah, so like, let's say if you're taking that approach, right, so you have a monolithic app and you're rewriting it in a microservices-type architecture, and you start breaking it into pieces. So you say like, each function's gonna be its own kind of microservice within this app. So then as you start breaking it apart, if you notice some of those microservices basically don't do anything unless one of the other services calls it or someone in the app does something, then that, then you might pull out, you may say, you know what, out of my 20 microservices that make up this app, 10 of them don't do anything unless it gets a call from something.

Well, let's move that to some sort of event-driven platform. And then you just keep moving them there. I mean, there's talk, and it really depends on what your app does about being able to move most of the app to this type of infrastructure. But obviously, there's some stuff that doesn't fit the model. Yeah, some stuff just needs to listen. You know, a database doesn't fit this model. You can't have a event, well, I don't know, maybe you can, but you can't have a event-driven database that you host yourself.

That's not a reasonable thing that persistently or has to always be there. Sure, sure, and when you really get, you know, when you say, like, what is this black magic? Like, what is Amazon doing? Or, like, for example, IBM announced a thing called OpenWhisk, which is a open-source event-driven computing solution. So IBM has a version of it in Bluemix, but it's also open-source on GitHub, so you can pull it down and set it up yourself and all those kind of things.

But under the covers, what are these things doing? They're taking, basically, containers and running them on demand, because a container can boot up a lot quicker than a VM. So that container boots, it's fed the code, it runs, it takes the output back. So there's some challenges around that when it comes to, like, say that one microservice in your app isn't used very often. Well, it may not be cached onto the node that it gets called for. So you may hit that piece of code, and you know what, it may take, you know, 200 milliseconds for it to fully boot up, to cache a version of the code out there, fully boot up, run, and then pull it down.

So that may, if some other parts of your app are latency-sensitive, that may not work. If other parts are busier, it's already cached on all the nodes, it responds super quick. It's some of those kind of underlying things that you don't really care about the individual infrastructure, but it's some of those challenges you may need to work into your app to deal with that. Or you may say, for this piece of code, it's so latency-sensitive, we want it running all the time.

But all this other stuff isn't as latency-sensitive, or it's called enough where we can run it in this kind of event-driven serverless model. So if you have a transactional-based application, you probably want those services running at a minimum somewhere. However, if this is one of the many batch processes in a transactional-based service that's no longer in the critical path of having the transaction committed, then this is something that, obviously, you can leverage and scale. Yeah, and then it's always that trade-off, too, of one of the things we've seen people go overboard with things like microservices, like, hey, I'm a new technology startup with this great new idea, and we're going to start a company.

Well, let's write our app in 20 different microservices. Initially, you might be just better off writing a big monolith code. Your time to value is much quicker to get to your MVP, to get investors and customers and move it along. And then once you hire a couple more people, it'll start to break it apart to microservices. I think this is something along those lines, too, where, hey, you're building, you know, that microservice is eight lines of code. It may not need to be its whole own separate event-driven service.

Unless there's specific value, and I think your photo bucket sharing app is a great example because that scales, from an architectural perspective, that scales quickly and it makes sense because the bulk of the traffic and your S3 object bucket is getting hit pretty hard and a service like Lambda can scale really well with that type of example. But like you said, if it's a bigger project and it's eight lines of code, it probably doesn't make a whole lot of sense to break that into a microservice initially.

Yeah, so if your main app is running and then you're going to call this one function to add a user to your system or whatever, and it's eight lines of code, you know what, maybe it's just sitting inside that same code base, that function, and then we'll go from there. Why spin up a whole nother little instance of the code in a container somewhere just to execute those eight lines? So I think it's going to be one of those, you know, finding the happy medium based on what the individual use case is.

And I think this lends the conversation. I really liked that you went through kind of what happens in the back end, that these are containers that's getting spun up and executing this micro-code, because this is something we can relate to, to the private and hybrid data centers. We look at solutions like OpenStack, cloud management platforms that provide the compute for that. I can start to wrap my head around, okay, what does that look from an orchestration and a container management piece? So I have this platform, let's say that it's Bluemix with the OpenWhisk sitting inside of it.

So is OpenWhisk communicating to a container management or orchestration tool, or is it communicating directly to OpenStack? What's the flow of communication? Yeah, so there's, each one's, you know, unless I missed it, Amazon hasn't laid out exactly how Lambda, how they're executing Lambda on the back end. We do know it runs on EC2 on the back end, we just don't know kind of the particulars. OpenWhisk, there's, I think it's even in the GitHub page for OpenWhisk, has a pretty decent architectural diagram.

It's using a lot of different open source components like Apache Kafka and other things to be able to provide that communication layer to be able to scale and build out these instances as you go, right? So you have to have the compute resources available to you and then the orchestration to be able to call them as you go. Yeah, because obviously there's the concern around load balancing these containers across multiple container holds, et cetera, et cetera. So that's something that I got to make a note to go and take a look at the OpenWhisk GitHub page and peel back that onion.

That seems like some pretty interesting sausage making stuff. Yeah, yeah, I mean, it's those, one of those things is you don't want to think about it while you're eating the sausage, but it is nice to know every now and then kind of how that sausage gets put together. So this has been a really enlightening conversation. You working on anything or you have anything coming up that you want everyone to know about? Where can folks find you on the whole Advocate trail?

You making any special appearances? Yeah, the Advocate team is going to be pretty busy in the next couple of months. We have the OpenStack Summit coming up pretty soon. And then it seems to be one thing after another, OSCON and let me see who else, Cloud Expo East and Cloud Foundry Summit. And what I think is really, really interesting and really cool about all of this is just kind of how well IBM and the IBM Cloud team kind of, we've been integrating together to deliver these types of things.

I know even early on talking to IBM people as, the acquisition happened and was announced in June, it happened in June. And then the fact that we've been launching new products since then. Usually it's for a lot of acquisitions, it's kind of business as usual until the transfer business. But for us, it's been, we had a new release in November. We moved into SoftLayer. We announced our and started shipping our local product. We have a new release we just announced last week with HIPAA.

So if you need a HIPAA private cloud in a SoftLayer data center, we can provide that. So there's a lot, there's constantly new stuff coming out. We work in that kind of continuous integration, continuous delivery model. So it's great to just see constantly new things coming out and be able to not just talk to people about it, but then also with the broader open source communities and how can we integrate these pieces together to really deliver actual business value besides, hey, we can create some VMs and some networks, big deal.

Yeah, I think I'm really impressed with IBM having kind of a couple of years ago, a very nondescript cloud strategy to having the SoftLayer acquisition and then building everything on top of SoftLayer, you guys, and even if you're more of a legacy organization, their vCloud Air solution, which again, I won't get into, but having that repeatable process and automation built upon SoftLayer, IBM has had a very, very good cloud story, they say, to tell over the past year and a half, and it's been an impressive transition.

So where can people find you on the web? com is my blog, or wherever else. All right, man, well, I really appreciated you taking your time on a, this was recorded on a Sunday night, so I appreciate you taking your time off hours to chit-chat with me. We'll talk to you guys next episode of CTO Advisor. com. You can find me on Twitter, at CTO Advisor, and please share us with your friends, rate us on iTunes, and share the love.

Thanks, talk to you guys next time.