Microservices in a hybrid world
Transcript
>> Hi everybody today I'm going to be talking about microservices in a hybrid world. First up just want to introduce myself, my name is Kyle Davis I am the head of developer advocacy at Redis Labs. You can find me at these two different places and I've been using Redis for a long time, but more importantly I've been developing applications and working as an architect and developer for many, many years and, in the past few years, I was introduced to the microservices architecture and it really, fundamentally changed how I see -- how applications are built, and it really does serve as a great way for many applications to be developed in a fundamentally different way from what they have been in the past and it's a very approachable format that does lead to a better, overall system, in my personal opinion.
Now, there is some controversy around microservices, I think microservices are one of the things that are oftentimes attributed to a variety of different ails, I'm going to go over how I see, and how the industry sees, the microservices architecture, and then how it kind of fits into the hybrid world. So, first off, I want to level-set a little bit here. Microservice doesn't equal a bunch of small services and I think this is the most common misconception that people have: that they're going to have a thousand services and they're going to be a hundred lines apiece and everything's going to be fine, and I think that doesn't really reflect the reality.
So, what is micro about microservices? Effectively, in a monolith, the responsibility is across the entire application, in that application everything is tightly wrapped together. Now, contrary to what most people think, this doesn't necessarily mean that everything is one binary executable, it can actually have many different executables. Really, the thing here is that there's a responsibility that is shared among all those executables for the responsibility of the application. And it really changes how you organize the code. Now, in the microservices architecture, basically what you're doing is dividing up the responsibility of the application into smaller chunks, and that's where the micro comes in.
So, in a monolith, you're actually organizing your code quite differently, and this is something that I've done in the past as I organize my code this way, or when I was building big applications. Basically, you would have the entire application but you'd have a section either that libraries or through modules or through even separate binaries or scripts that are responsible for different technical capabilities of that. So, for example, you might have a lot of different code that's kind of responsible for sending emails out in an eCommerce application or, for example, your rest API might be somewhere else than maybe, you know, PDF generation.
That is fundamentally different from how the microservices approach works. In the microservices approach, basically, what you're doing is you're transposing the organizational units of the overall organization onto the architecture of the application. So, in this case, let's think of a kind of mythical eCommerce application, and some of the services you might have there is you might have a fulfillment service, you might have an order service, and you may have a user service, and those kind of reflect the organizational view of fulfillment, order and user, whereas, they may have many of those technical capabilities all in the individual services.
Now, I want to be pedantic for a moment, you'll hear people say, "This is my fulfillment microservice" and, for people who have studied this and seen how these ideas came about, that may make your skin crawl because the preferred nomenclature is actually fulfillment service. So, the microservices architecture is made up of services, this is something I put in all my presentations about microservices, I'm sure this particular crowd knows this but it is something that really does make a difference and, I think, actually informs how you think about microservices.
So, let's talk about some more kind of properties of this. Each service has its own centralized -- non-centralized storage, so each service has its own local storage that is available only to that service, so you are reading and writing on the fulfillment service to a single storage place and you cannot -- the fulfillment service cannot read and write to the order service. Now, there are exceptions that we will get to a little bit later but they're kind of siloed here and each individual one is responsible for its own storage.
Now, the other kind of property that we have here is that services are stateless. Now, what do I mean by that? In this case, it means that each individual instance of the service does not keep state on its own, and that's actually kind of important how it works the storage too, so take for example a fulfillment service and a situation where one of the instances dies, right? It can then be brought back up and replaced without having to worry about anything, and this kind of goes to the, you know, cattle not pets part of some of the modern thinking around servers and services, but what's really nice about that is that you are always just reattaching to the storage that you might need, the database that you might need.
The other thing is that you can actually have many different instances of a given service and, because they are stateless, it doesn't really matter. So, instance seven of the order service is still attached to the same storage as instance one, so it doesn't matter if instance one or instance seven is responding to any given request. Now, the other thing, shifting gears away from statelessness, is talking about the interface. Now, this is a little bit of an overloaded term but really this is the interface between different services, what I'm referring to here.
Traditionally, this has been something that would be like a rest interface that runs over the HTTP protocol or the HTTPS protocol. Now, this is not the only solution, which we'll get to a bit later. But basically, when you're doing this, one service is talking to another over kind of the standardized interface, and the key with the interface is that it is the stable part of any given service so, in that case, let's just say that you have some problems with your order service in the way it was built was not up to snuff, you can entirely replace that as long as you keep the interface, how one service is talking to another, the same it doesn't matter underlying what you do to it.
So, you could go in and have something implemented in, say, JS and say this doesn't perform, then you can replace it with something of an entirely different language or change the underlying data store. Now, a single operation may involve many different services. Now, I live in Canada, so we're going to use the example of purchasing a hockey stick. In that case, what we're going to be doing is actually looking at how that might work and what actually happens underneath the covers when you purchase the stick.
So, in this case, you have many different services talking to one another to accomplish the same role. Now, this is where the rubber starts hitting the road when it comes to a hybrid deployment, what happens? Now, these are all synchronous requests, one to the other, what happens what you cross maybe a boundary between public and private cloud, you know, you have your on-prem fulfillment services, and then public cloud is the other two services. This really does have some implications. Now, what you really have to ask yourself when you're doing this - you're setting up an architecture like this, is are these the same?
So, in a public cloud, you have these two services that are talking to one another and you have certain guarantees and certain things that you take for granted, certain things that you don't take for granted, and maybe you're working around. Whereas, when you cross the public cloud-private cloud boundary, what does that mean? Is there different guarantees there? Is the latency the same? Oftentimes, these are vastly different types of connections, you would have to consider things like security, and, you know, are you in the same rack maybe?
That's something that you can have very fine control over and have shorter core physically on-prem situation and in a public cloud situation, you may not have that control, or when you're crossing the boundary maybe you don't have the same guarantees about stability of those connections, and how does that impact an overall application build in a hybrid world. Now, there is an alternate way of viewing this, now, we have assumed HTTP up to this point as the communication method and a synchronous communication method, at that.
Let's start thinking about things in a little different way. Now, we've talked about interface being key and all this stuff and private, you know, databases and things like that. Now, we talked about the -- I mentioned earlier about the exception and let's talk about it now. In this case, you have the database for fulfillment reading and writing to it, but if you have a situation where you have another service reading from the database of another service, what's the key here? Why is that different than anything else?
At that point, that part that you are opening up for other services to read can be considered an interface, and when you have that you just have to keep it stable like you would -- just like you HTTP interface, but you can only read, and read is a non-destructive activity functionally and so, in this case, if you do want to make changes, you have to go through another synchronous form of doing this, another interface, you can't just write to the other database, that would violate a lot of the principles of the microservices architecture.
But what does it really mean with your doing this, you did a write over here, why wouldn't that be a good idea? The reason is you start generating very tightly coupled applications, also there's a risk here if order service makes a change that fulfillment service isn't expecting then you have the situation where you might as well put those two service together. Now, I want to talk a little bit more about this but I'm going to shift gears for just a moment and talk about Redis.
Redis is an in-memory open source database, for those of you who don't know. We're assigned for a lot of different use cases and it's pretty widely known in the industry but there's a -- people tend to underutilize Redis so I go over a lot of this in any presentation I give in front of people. It stands not only for Remote Dictionary Server, that's a little bit controversial these days, on the ten year anniversary Salvatore, the guy who created it did let it go that basically he was -- that was a bit of a backronym and he is really looking for a five-letter domain name, so it really did have a little bit of a strange origin to the name.
Like I said, it was created by Salvatore Sanfilippo a little over ten years ago, it's open-source BSD license, and I encourage anybody to look at the source code, I think that's where the real proof of any technology is, if you can read the source code and say this is really good code then you can use it. Now, I work for Redis Labs and Redis Labs is the home of Redis, but certainly also it provides Redis Enterprise and Redis Enterprise is an in-memory multi-model database built on top of Redis, but what it's really used for is it provides ways to enhance Redis, of course, but it allows operations of Redis to be a lot simpler for people who use a lot of Redis.
So, if you're a small organization and you use a ton of Redis that's fine. Redis Enterprise is for you. Or if you're a large organization and by function of that you use a lot of Redis, then Redis Enterprise is certainly for you, and if you're, frankly, building a microservices architecture, probably Redis Enterprise is for you as well. There's very few true small microservices protectors out there and Redis is usually a key part of most of them. Now, from the data model point of view in Redis, there's a -- we descend from this whole idea of a key value store, and key value stores have this idea of a key and some sort of data on the other end of it and that's about where it ends.
Redis has extended that and basically now you have another layer which is the data structure layer. The data structure layer kind of informs how the data is stored, and retrievable by a given key and command. Now, I'm not going to talk about all these different data structures and, in fact, you can expand this with modules, but what I am going to talk about today is Redis Streams. Now, Redis Streams is a series of time-ordered entries in a database, we'll get into it a little bit more but you can think of them as field values that have a way of being intrinsically ordered and it really provides a way for you to go back and look at something in a mutable way.
There's other ways you can do this: you can implement this type of data structure on your own, obviously other services that do this, but I will talk about Redis today, this is what I know. Now, let's kind of look at this from a very high level. How does this work? So, you can give many services asynchronously reading from a specific given stream, and the way it kind of works is you have your one writer to the stream, in this case -- woops wrong direction.
You have the user writing to the stream, and then you have other services reading from that stream, so we're still using our fulfillment and order service here. So, the key thing is that they're keeping track of their kind of last seen. Now, everything in a stream is time-ordered and it has a time-stamp and a sequence number and it prevents any kind of duplication here, but I've kind of reduced this down to minutes for this kind of an example. So fulfillment, for example, has a last seen of nine-oh-two.
I kind of think of this as like a bookmark almost, so fulfillment knows that they left off last time they processed something was at nine-oh-two, so they can then read from the stream and say give me everything, or give me a few entries, past nine-oh-two. It will do that, it processes something and then it updates its last seen, and then it will loop back and do it again. Order can have a separate bookmark for it, so, in this case, it's nine-oh-four and they say I'm going to wait for something to come along and when it comes along I will then process that and update my bookmark, and that's how you do asynchronous communication among different services in microservices architecture.
Now, let's dive a little bit deeper into the data structure itself and how that has implications on things. So, you have a series of different ones, in this case, the way Redis Streams works is you have fields and values, and they're kind of ad hoc, you can put anything you want into them, so you can have different fields or different values in them, as long as you're kind of keeping the same commands for a given data structure, in this case streams. These are called entries so any time you create a new entry at a given stream key.
And this is something that you'll see the time sequence is automatically managed by Redis. So, diving a little bit deeper onto this, how does it actually work? So, you have your stream key, it has some data in it. We have one process which is adding to that, so it's going to add the stream entry, the field and values that you put into it, it will assign a sequence number and timestamp, and that will be the stream ID. That stream ID is then, you know, returned back to the application and it can do whatever it needs to do with it.
The reading services here will do what's called a request and read, executing request and wait, which will basically, using the command XREAD - that's the Redis command, you don't necessarily need to know that to understand it - but what will happen is it will take its last known entry ID and then say give me everything for that. Now, if nothing happens, it's waiting, so in this case it's kind of a push model, so it's holding open saying, for a given amount of time I'm going to wait for anything to come in, so when it does, it will return data, and that data will include the fields and values that it had as well as a sequence number.
So, then what will happen, it'll process that and it'll acknowledge that, which we'll get into in a moment after the processing has occurred, and then it will immediately start saying, my last known ID was the ID from above. So you can see here this thing that just came on the screen, this is the last known ID. So, as we move forward, we will then process that again and keep on going. Now, if there's a timeout on this and nothing happens, it's just going to keep on pushing in that value and it can wait, let's say, up to a second, if you want to put it that way, but you can specify that in a little bit different ways.
So, this is actually quite important to understand that as you do this it creates this loop and it will never be able to lose data because it's constantly keeping track in its own data storage - the services data storage - where it kind of left off on this. So it actually becomes quite resistant to a lot of failures. Now, let's talk about that in a little more depth. Redis, like I said earlier is an in-memory database which has some interesting things as far as durability is concerned.
This is where a lot of people, kind of, raise and eyebrow when you're doing these type of things, but this is something that you'd want to look for in any data storage that's doing this type of workload. So, in this case, you know, your service is writing to Redis to a stream and the command here is XADD, you can see that on the screen here, and it's allocating memory to do that, and then, after the memory is allocated, it's going to append it to what's called an append-only file, AOF for short.
Now, Redis has a lot of control here to do different things. We want to fsync that in a lot of different ways. Now, fsync can occur three different ways for Redis, and one way is optimal for this kind of workload, and you can fsync every periodical known time, so every, you know, several seconds, thirty seconds, something along those lines, and then basically that means that anything that hasn't been fsynced is potentially not actually committed to disc, so that has some drawbacks. The other option is to use fsync on every write, so Redis keeps track of where the writes are and, in this case, you really want to make sure that if something is committed to this that it's an action that is super dependent on other actions, this would be really important to do this, so you're going to want fsyncs on every write, so once you do that XADD it's going to say, yes, I committed it to the disk.
And the third option is kind of more loosey goosey and that's just based on the operating system, so it may fsync at variable lots of time, but for this type of operation, you want fsync on every write. Now, in a success situation, it's going to return the stream ID back to your application. The key thing here is, though, if there is a failure that occurs in any of these steps, it will return an error, and notice that what's happening here is this is the last part of the actual information, so let's say there is failure of the entire instance you get a failure scenario that way, if there's a failure of the disc or the memory, you get a failure back off the client itself, so you'd know you'd have to do some sort of retrying method.
So, that's really important in getting -- making sure that when you're writing these things that it is done in a very durable fashion. Now, looking at different things from how you might do this, of course, in a place where you have multiple instances, of course when you have very few actual architectures have just one instance of something, right? That's kind of the dream, you have multiple instances handling a load. So in this, for example, we have our order service here, let's say we have seven instances of it.
You need to be able to manage this across entire groups, so just keeping track of that bookmark is not sufficient. You need to be able to keep track of that bookmark as a group. So, what happens is, you know, this order seven instance is reading this stream ID, right? And it takes some time to process this. At this point, there is a group that's maintained by Redis that is making sure that it's not giving these things out willy nilly to multiple different instances if that's your requirement.
So in this case, what it's going to do is order seven is going to be doing whatever process it needs to do, sending emails or, you know, doing some calculations, and to prevent duplicate work, it's going to do that and not pause the database in any way, but after it's completed, it will then acknowledge that it's finished with that, and we use the XACK commander, ACK just for short. So that's all fine and good, but that actually opens up another kind of hole, right?
You want to be durable to any holes, or resistant to any holes that you might have. What happens, though, when you have a failure in that particular instance? So order seven dies for whatever reason. In an on-prem environment, maybe that's something like you have some sort of disaster that occurs inside the data center, or maybe you need to do maintenance, or something along those lines. You have to yank the plug sometimes. Now, in a public cloud you have other scenarios that are even more unpredictable, you might have some sort of interconnectivity failure, you may have some maintenance windows.
You need to be able to be resistant to almost any failure. Now, there's a particular pattern that you have to do when building this application to make sure you can do this. Now, in something like kubernetes which you would be using as a stable set to keep track of the kind of names of the individual instances that are running, so in this case, when it dies and you bring it back up it will know that it is order seven, and then what you do when you start up an instance is you just say, give me all of the things that I haven't processed yet, and 00 is the command [inaudible] that you'd give here.
But it's just important to know that that's there to make sure that you read back through anything that you would have missed, and this enables any instance to go down at any given time and actually be able to make up anything that it has already reserved, and not duplicate the work if you need it. Now, I've given similar presentations on this topic many times before and some people gave me the feedback and walked away with an idea that I didn't necessarily want them to walk away with.
Synchronous operations are certainly valid in a microservices architecture, so if you have, for example, order and fulfillment doing a synchronous operation over HTTP there's lots of good reasons to do that. So let's give an example. So order sends some sort of rest request or rest API request to fulfillment, order is now waiting for a response, and it needs to wait for whatever type of, you know, rules that might be in place for a given piece, so it needs to get an okay from fulfillment, so in this case, it is having to do something else, which is like, for example, update a status in the private database for order.
You wouldn't want to do this in an asynchronous format, this would be possible but very challenging to really master doing this in any reliable way. So that's certainly something you'd want to do. So synchronous operations are still something that you have to do, but using asynchronous operations in a stream is also something that is valid. You really have to consider what would work best for your given circumstance. So for example here, user is listening to order, listening to a stream, right?
Then, order adds this new order to the stream entry and then it sends an email out. In this case, you're not having that back and forth, right? Sending the email out certainly is something that you don't want to fail on, but it's also something that the order service doesn't need to wait around for anything to happen on. So certainly both will make up a more complete picture of your microservices architecture. Now, I do want to talk about another kind of component to this which is the microservices architecture in latency.
And this is really important in a hybrid environment, especially considering that latencies between services may be longer than they would be if everything is on-prem or everything is in a private cloud. So your overall latency really is the sum of latency for all of the different services in a given operation. So if you have higher latencies, that becomes a problem very quickly. So let's take a look here, you have fulfillment and it does all its database operations in twenty five milliseconds, and you may think, hey that's great, that is a very respectable figure for many databases and you think, okay I'm in good shape.
However, things get worse when you have this across many different instances of a given, or many different, services and many different instances maybe even inside of this. So that actually becomes a very high figure, so it's just this five different operations -- five different components of the operation we're doing is a hundred and twenty five milliseconds, and that is probably intolerable for the overall user experience, so we're looking at something, we like to talk about the instant experience in Redis, and that's anything less than a hundred milliseconds is something that you can't really understand as a human being, but here alone just five different services in a row doing something relatively fast becomes quite slow.
So what do you do? You can still have your database accesses twenty five milliseconds, but putting a cash layer in is something that can really make a big difference. In this case, you have a cash layer than can respond in a single millisecond and, even though you're doing five of them, you have a still respectable five millisecond overall time. So that works out well. So not only can, you know, you really start thinking about how you might communicate as well, but the time in which it takes you to communicate is incredibly important, especially as you think about these arrows being variable on time.
Now, I do want to plug something here: my book that I wrote with my colleague Loris Cro. This is Redis Microservices for Dummies. Much of the material you saw today comes from this book. ly/cto-microservices to pick up your copy for free. Like I said, it's a quick read, less than a hundred pages and it goes over a lot of these things in much more detail than I could give on a thirty minute presentation. All right, well I want to thank you for the opportunity to talk about this, show you my book, and I'll be around for the Q&A session so let me know if you have any questions.
Thank you very much. [Fiery, Electronic sounds]
