Modernizing Legacy Applications for the Cloud: Lessons from Granite

Machine-generated from the episode audio and not hand-corrected, so names and technical terms may be imperfect. The audio is authoritative.

All right, you're watching. I don't know what to call this. Is this a podcast? Is it just a CTO dose? Who knows? The important thing is I have Alastair Cook, my fellow CTO advisor at the CTO advisor, a future group company. We are asking the big question. If you want to move to the public cloud to take advantage of cloud based models. So a. A. W. S. Sage, what is it, SageMaker, Sage, SageMaker, SageMaker, OpenAI in Azure or Vertex or Google AI in Google Cloud.

M. M. We've known for years that you just can't lift and shift and expect cost savings or even to have on par on par cost. M. to some type of native cloud environment, whether you use a native cloud services or containers. They have to run in such a way that are economic, that makes economical sense in the public cloud. So, Alastair, can you explain to the audience what we have or what we had yesterday and what the vision of what we wanted to take it to?

Sure. What we have is an application that I've called Granite. And the objective with this application was to look like an enterprise actual application. So it's an order entry system and fulfillment. It has a web front end. It has a MySQL database as the center of it. And then it has a series of workers that behave like a bunch of users or whatever the business function is. So in the front of it, there's an order entry system where there's an automation in it because I didn't really have 300 staff to go and run this thing for me as a production application.

There's an order entry component that takes a list of our products and a list of our customers, and it orders a whole bunch of product from that goes into the database. And then there's a series of processes that analyze these orders and say, do we have the stock on hand? Do we have to order it from a third party supplier? Do we have to manufacture it ourselves? And it goes through those series of transactions, eventually fulfilling the order and saying, yes, we've brought together all the parts.

We've assembled the thing and we've shipped it out and then generates an invoice. And so this is a distributed application that runs across somewhere between eight and 10 or 12 virtual machines and was designed to run on premises on a vSphere infrastructure. So it's a bunch of VMware VMs. And fundamentally, that led to some expectations about how you build your application. You've got this virtual infrastructure. There's a very small incremental cost to deploying another VM on it. And in order to get that containment of resources, we just deploy another virtual machine, another virtual machine for a load balancer, another virtual machine as an additional web server, more worker nodes if we want to generate more workload.

And so there was a whole collection of things that are pretty normal things to do when you're building an application on premises, including things like using NFS file share as a location for those those invoice PDF files to live so that they're accessible across the web service that make up my behind my load balancer is the web interface. Yes. So, again, it doesn't matter why we made the architectural decisions. This is the application that we have and this is the application that's actually served us well for the past few years to do all kinds of testing.

And as you're listening or watching this, you're imagining, well, this could have easily been a containerized app. It is built on a set of services, but it's not. And we made design decisions that made sense for our virtualized environment that we had at the time. It didn't make any sense for us to run out to roll out a Kubernetes cluster or even a docker container cluster. All our other applications were VMS. We VMS are incremental in cost. They're basically free just as long as we're not expanding beyond the capacity of our fairly robust lab environment.

You know, we have two servers with a terabyte of RAM. So each so that, you know, we have plenty of headroom. So it didn't make any sense to complicate our environment by going for control plane that none of us knew how to manage. So that's how we get to where we're at. I think now it's really important to recognize that's also where a lot of customers are at. A lot of customers have built a lot of built business processes into these VM based applications that were typically three tier applications.

And so they are built with lots of virtual machines. And fundamentally, that's why we started here is because we wanted to reflect the journey that real production environments are going through and enterprise organizations. Yeah, and that's the unique thing about the CTO advisor. Now, the signal 65 data center, right, that we mimic a customer's environment. We started out with legacy hardware. We modernize that hardware. We modernize the VMware stack. We went through this process. We connected it to the public cloud.

We were able to be motion and expand our VDI infrastructure from the private data center into the public cloud. Once, you know, the everyone was working from home, then we could right size that again. We had a proper or have a proper hybrid infrastructure. And we've done some initial works, workshops and some initial research with Google and identifying and migrating these workloads to the public cloud. But without a doubt, I think we've all consistently experienced that simply lifting and shifting this stuff like for us to lift and shift this.

These workloads to the public cloud in VMS would be cost prohibitive. Like my AWS bill right now is thirty five bucks a month and is thirty five bucks a month, because the only thing that we have actively running in AWS right now is our S3 stores to back up for vaulted backups. When we need to do experimentation, we use AWS for what most enterprises use AWS for to experiment. And then we bring that stuff back on prem. However. I. I.

in general is different. It is very much in the experimental phase of which you want to iterate in the public cloud. Figure out what you're going to do, what the app is going to look like, what the workload performance and resources will need. Then we can make an educated decision on that data that we get from the public cloud and our POC's in the public cloud and either decide to keep it in the public cloud or bring it back on prem. And the one thing that we can't really can't emulate on prem is the speed to value for using some of these services.

You know, Google Cloud did a really great demo of using Cloud Run to leverage their infrastructure and just make API calls to it. So if you kind of went over Alastair's app, we have we can create three hundred workers that do imitation order entry. I. environment that replaces that that function, that simulating something that's a little bit closer to a real customer load. Yeah, I think there's also in terms of delivering some business value to what what we're trying to do in this is using the sales history for a series of customers to identify what products we're likely to sell and to identify basically getting ahead of ordering stock for those products, because often in the sort of flows that I set up with with Granite is that there's a big latency between the order being placed and the product being produced because we've got to wait for manufacturing.

We've got to wait for order out. I. to do prediction of what those orders are and to improve the just in time delivery of those components. There's then also the generative idea about getting customer specific advertising. So as we're starting to bring in new new products, how do these correlate and how do we get customer specific information out? So although the actual application I've built is relatively small, the applicability for adding tools into your application is much wider. The other thing that I think is really important is that the tool set for building a general application on these public cloud platforms is far more mature than the tool set for building the same thing on premises.

I. solution on premises is really high because the tooling is just here's a pile of screwdrivers and hammers and bolts. Whereas on the primary public cloud platforms, here is the actual as a service delivery. I. things on cloud. And, of course, they're dependent on having access to our data. That was the important thing about the demo of Cloud Run doing rag retrieval into generation that we saw at Cloud Field Day was that the knowledge is in our own data.

I. to attach to. The sort of natural language processing part of the foundation model is awesome, but it's locking it to your corporate, your internal data. I. Yeah, so we we can't go from a model in which we have eight always running VMS in the public cloud. No, that will, again, be cost prohibitive to be at least at least in our use case, at least three hundred. I mean, eight hundred dollars a month with the smallest VMS available in AWS today.

I think I clocked it about one hundred and twenty hundred thirty dollars a day to run, you know, really small VMS in the public cloud or one hundred thirty dollars a month to to to run those instances. So, you know, we we go up to a thousand dollars a month from, you know, no no VM build this this month to a thousand dollars a month. And Keith went in our CFO and I have to explain that to him that I don't I don't want to have to.

So with that said, we see the problem ahead of us. We have the app on paper. This app is extremely contained and friendly. Also, you've built, you know, kind of this the this set of apps with services that could easily either be. We could either replace them with services one to one like my sequel. I can literally just create a my sequel instance in the public cloud and use that serverless instance. And these run times, these containers I can run and EKS cloud run and whatever Azure names their thing.

But first thing we have to do is containerize the app. And that's what you've worked on for the past couple of weeks. Talk to us about that lift. Sure. Well, I did originally write this to be relatively easy to move across into containers. So I knew this was was coming and I didn't want to make too much pain for myself. So each of those little processes that create an order, produce a part, order a part fulfilled. These are all just self-contained Python scripts.

And so getting those into containers was very simple. There was the Docker file to build those containers is about six or eight lines long. There's a few dependencies for some of them and generating a PDF. I've got to actually pull in the right libraries into that Python container. So getting the actual business logic code in was relatively easy. I did hit a couple of snags. One is that I have hard coded hard coded the DNS name of the database server into my Python code.

And I'm realizing that, you know, there is a reason for using environment variables to pass these things into containers. And maybe I should be following that best practice design. I've never said I was a good developer. I also never said that I learned the lessons that I taught. So teaching AWS training courses about building applications are absolutely build your application for the cloud. You're using containerize, use environment variables, separate out config from persistent data from execution. These are things that I'm now having to retrofit partly.

So the kind of things that are challenging is that I've used a file share for those PDF files. And it was just a file share on the one virtual machine that generates the PDF file. So it was just running an NFS server. And that NFS mount was then mounted to the two web servers. So they got access to the same thing. Well, containers, that's not a sensible thing to do. Happily, at the moment, it's just running on a single container host.

And so I can just use a file system as a volume. But I'm going to have to work out what's the right type of volume to present to both that invoice creator and those web server containers when this gets moved out onto a distributed environment. But the big challenge is that with those virtual machines, I had dedicated resources that I had no incremental cost to use within the virtual machine. So in each of those worker virtual machines, there's a schedule task that runs.

And it runs in a fairly tight loop. It just checks the time, reads the database and says, is it time for me to run one of the processes I'm responsible for? Now, that's okay to do inside a virtual machine, particularly if you actually go idle in that loop and then release the CPU to be used by other virtual machines and then come back maybe a second later to see if there's something to do. But having every container running all of the time in order to run that schedule doesn't seem to be a good idea.

And so the thing I need to make more native is that scheduling. And have the scheduler talk to my container platform rather than just launching local processes. That's the bit that I'm looking at at the moment. My first pass is that Kubernetes has a jobs function and a cron job function. So you can say every minute run this container or run this pod. However, every minute isn't fast enough. And so I need to write a scheduler that will operate at the second kind of rate or preferably tenth of a second kind of pace.

And that's the next hill I've got to climb building that. And if I'm going to do this on Kubernetes, then it should be an operator and it should be a container that's designed just to do this and then integrates with the Kubernetes API. Like Kubernetes for that because, of course, it's the same API whether I deploy on AWS, on Azure, on Google or on-premises. What I saw was that I built this application a while ago and there's new versions of Python and there's new deprecation of formats.

And I did have to make some code changes to update for the new version of Python. Although that's a pretty normal kind of thing as you're updating Python applications. So, you know, the I think we're highlighting a couple of problems when you're going from one architecture type to another, always on VM running. This is one of those things that I really had a really difficult time wrapping my head around. And and traditional monolithic applications is running on a VM. The computers running all the time.

So the cost of having a daemon that was constantly listening was minimal. That that was just, of course, the VM would be on. Why would it not be on and need to re jigger my mind to this idea of serverless, where one, I can schedule something to run or I can make it event based so I can have an event bus and I can listen out for an event to happen. And once that event happened, I can pick that event up, triggers a container being ran.

It runs for a period of time. But again, your your application logic has to lend itself to that type of thinking. Right. And then from an organization perspective, you have to think about event buses, which a lot of us have. We have Kafka. We have well, we don't have Kafka, but a lot of organizations have a Kafka. They have one of the more legacy event bus solutions out there and they can build around that. And but it takes this, you know, rethinking of the application logic and these dependencies that you just weren't aware of because it ran in a VM that ran all the time.

Yeah. And this is fundamentally the sort of the center of the idea here is that when you build an application as a developer, as an application architect, you're building it on a set of assumptions about how the infrastructure works. And the on-premises assumptions are very different from the on-cloud assumptions and lifting from one place to the other, replatforming, redeveloping your application is a big uplift. And to get the most value out of that cloud platform, you really do have to start again with a new set of assumptions.

And that comes back to that whole idea of cost, right? Is the cost and the time to do this going to actually get us a return when we get this thing into public cloud and we're running this adjacent to whatever other services we want to consume? And that fundamentally is the whole point here is to get that business. And, you know, I have two minds of the effort, the whether the effort is worth the the juice, if the squeeze is worth the juice is one thing is this is known, right?

We know how to do this. This is moving legacy applications from the private data center in the public cloud. This is not rocket science anymore. We you know, there's no known patterns. There are solutions to these problems. So that lift, while labor intensive, is a known lift. On the flip side, standing up a eye in a private data center is a unknown lift like. I had a conversation with Gene Kim last week and he was talking about spending a couple of hours trying to load drivers for GPU.

I don't want to pay a thousand dollars an hour for my developer to try and load drivers that I've had there, you know, two or three hours of mucking around with the environment, the infrastructure. I could have paid a month of experimenting in the public cloud and getting real value. But, you know, kind of a sub factor to those two factors is, wow, I can get addictive to cloud in both ways in the obviously in the way that I'm, you know, using those runtimes in a way that I can't easily recreate on prem later on.

And then, too, as I'm building services around the public cloud, I hinted to, you know, you can we can do a MySQL service and I can consume that. I can recreate that on prem. That's not too difficult. But the service bus part. That is where we can get addicted into a certain cloud provider. So, you know, some people might just be asking, well, why don't you just use the AWS service bus instead of, you know, going through the scheduler problem?

Well, now I'm creating a dependency on the public cloud. And now if I want to move this application back on prem, I have to figure out how to either mimic the API call or I need to find another solution to bring the application back on prem. And these these cloud comforts have a way of creeping themselves into our and their cells and themselves into our architecture, just like the VM comforts creep its way into our architecture. Yeah, I think one of the things I definitely saw in AWS, since that's what I've specialized in, is that it's a developer enablement platform and it really does bring a whole bunch of easier ways than having to go back to the infrastructure team and get those drivers loaded.

For the GPUs and get them licensed and all of those challenges. And it goes right through, you know, having a message bus, having tools like Lambda, so functions as a service or on Google using Cloud Run. These are all the sorts of tools where you get a lot of value, but because you're getting a lot of unique value out of the platform, you're going to be tied to that platform, you're going to invest development time, and there's going to be a cost to switch away.

So it's a good thing because we're getting lots of unique value, or it's a bad thing because we're making the cost of switching much higher. What has been your experience from two levels? One, the experience for migrating? Granted, how much time have you had into this application and containerizing it? And then two, if we had to get the factory mode, we're doing one hundred and twenty of these apps. What's your sense of the level of effort when it comes to creating a modernization factory?

And people are going to ask the question, can AI help in this process? So the first side, this relatively simple application, I got all of the functions running in containers in about two days of effort, and then getting that to translate into running the application on Kubernetes is taking a bit more time because of that scheduling and getting things to run at the sort of cadence that I wanted them to run. In terms of the wider thought about migrating applications from on-prem to cloud, there's definitely a big uplift to doing it well.

We know that lift and shift is not enough. You've got to improve your applications either after you've moved them up into the cloud or before. You've got to make them work in the cloud with cloud services natively. And so there is a lot of effort in there. But there's a lot of patterns in there that are fairly consistent. And as you say, this has been going on for quite a while. People have been doing this for years. And so there is plenty of prior art to leverage.

Generative AI tools, AI tools integrated into development environments are very beneficial for this. It's simple things like being able to get on AWS Code Whisperer or Code Genius on Google to make suggestions. Here is what I want to achieve. How does the code work? So I've never worked with Cloud Run. And I can get suggestions about how to actually interact with the API for Cloud Run from this generative AI rather than having to go and read the documentation. And that's incredibly beneficial.

If I'm not going to specialize in this particular API, if it's just part of getting moved across, part of just onboarding, those generative AI tools that are giving me hints, it's like having an experienced buddy sitting over your shoulder saying, hey, here's a better way to do that. And without the cost, of course, of pair programming. So yes, there's definitely some value in those generative AI tools for assisting developers to get familiar with new platforms. But there is still going to be a lot of work, a lot of knowledge of how the application operates, how and why it was built this way.

And those are the things that are going to be a big challenge for enterprise applications that have been around for the last 20 years, where the people who made those decisions may no longer be with us, may no longer be in the organization. This is going to be tough stuff for large organizations. And coming back to one of the other things that we hit when we did the project with Google about analyzing what you have on premises and where it might land out on Google services.

It's not that you can do everything in isolation. It's that in an enterprise environment, everything is dependent on other elements. And it's that knowledge of the dependencies. It's the enterprise architecture and application architecture elements that are absolutely going to be really tough to get through and really important to establish very early on. Definitely get some experimentation out there, get some hands-on, get beyond just the intellectual knowledge that it is hard to change things to actually go and making some changes, getting the people who care and who are involved in building these applications to get that hands-on experience.

Because there's nothing like tearing your hair out at not being able to make the right API call to teach you where the right API call actually is. Yeah, some people may be. Looking back like they really don't understand the enterprise and why it's so difficult, and I think one of the things that you mentioned in passing is a great example. You have an NFS server. Again, this is not a production application. This is something that you threw together over the past few years.

But in a real enterprise setting, that NFS, while the developer is making calls to the file system. He has no he or she has no idea of what the infrastructure and operations team did to make that work like the VM, like it's a VM. Yes, but that VM might have RDMA storage backing and that RDMA storage might be exceptionally fast. So how do you mimic that in a public cloud is a regular easy to volume or NFS service in the public cloud going to be enough or are you going to have to subscribe to a higher value or higher throughput service?

All of these minor details matter when you're talking about migrating real production enterprise applications into services. This is all data points you need in architectural decisions. You have to make. And then there's different levels to which you integrate with the native services. So we're talking about using NFS file share and moving that from an on premises NFS server to an NFS service on cloud. But maybe I should be changing to an object store, particularly for this use case where I'm writing the data once it's not allowed to be modified after the invoice is generated.

But I might be retrieving that data multiple times, particularly if I'm using it as an input to my AI system. So maybe I should be storing this on object storage. But of course, that's yet more work. I have to do to make my application work with object storage rather than the file system storage that I was used to. Yeah, I would love to talk to our former colleague about this because he does this all day, every day at AWS. And I remember talking to him initially that, you know, they're encouraged to turn the knob as far as they can in modernizing applications.

And this is the challenge with the enterprise, right, is our and I recommend not making too many changes as much as possible. Matter of fact, I would actually recommend doing a V to lift and shift initially to make sure you get a production app that works and then taking the app apart once you get it into the public cloud. Because too many changes, too fast and you cannot support your environment. I've seen it, whether you're talking about data center to data center migrations, P to V, V to P really doesn't matter when your eyes get too big in the enterprise.

You find out how fragile your applications are. Yeah, the other element about it is that using cloud native services is much easier once you're already in the cloud. Now, the flip side to that is that once you've picked up what you have on premises and lifted and shifted into the cloud, your cost is going to go up. And you're not going to get the benefit of being in the cloud in that state. So there's some challenge then around balancing the investment of keeping the lift and shifted in place and the investment of making it more cloud native.

We do see progressive optimization as being a fairly common pattern. So we might move our entirely VM based infrastructure up into the cloud and replace the VM based load balancer with a load balancing service and replace the MySQL service with a managed database. And so progressively replacing piece by piece. And I agree, lots of small changes. This is the DevOps way. Make small changes, make reversible changes and very importantly, instrument as you go. So you identify when a problem occurs.

What was the change that led to that problem? Yeah. And so with that. What's next for us, Alistair? Are we what's going to be the next update to the audience? What will be the next major milestone for this project? There's going to be a couple of different parts. nz website some information about the Granite application itself. So you can follow a little more of the sort of technical detail, the hands on technical detail side of it. You and I will continue to have conversations about whether this makes sense and how you identify that you're going to get value out of this investment.

And I'm going to go and write myself a Kubernetes native scheduler in order to run these series of pods and at a higher cadence than I can get out of the cron job component. This, of course, is all waiting for me to finish fighting with some Oracle build automation to assist the rest of the Signal 65 lads team. All right. For those who want to find out more about the CTO advisor Futurum Group, you can go to Futurum Group dot com. That's the mothership.

And you want to find out more about the CTO advisor and how and what we're doing at this level. The CTO advisor dot com. We're we'll we are still there. nz or just demitas. Demitas on on socials and demitas dot co dot nz for my personal website. And I'm ironically still at CTO advisor, which is not confusing. The name of the business unit is the CTO advisor. The name of the company is the Futurum Group. And my Twitter handle is at CTO advisor.

And Alistair is also a CTO advisor. That's the only thing that's confusing. The we did not. Well, I did not think of this naming convention. Well, when I first named the company until then, again, check us out online. You have questions. Comment below. Hit us up. My DMS are open and I'm always available, of course, via LinkedIn. Thanks a lot, Alistair. Thanks, Keith. Good tracking.