Modernizing Legacy Applications for the Cloud: Lessons from Granite
Transcript
all right you're watching I don't know what to call this is is this a podcast is it just a CTO dose who knows the important thing is I have aliser cook uh my fellow CTO advisor at the CTO advisor a futurum group company we are asking the big question if you want to move to the public Cloud to take advantage of cloudbased models so a uh AWS Sage uh what is it Sage maker Sage it is Sage maker Sage maker uh open AI in Azure or Vertex or Google AI and Google Cloud what happens when you have a bunch of Monolithic not even monolithic applications applications that run in VMS quite frankly your your model for operating in the data center has been to operate with VMS we've known for years that you just can't lift and shift and expect cost savings or even to have on part onar cost you have to migrate these VMS to some type of native Cloud environment whether you're using native cloud services or containers they have to run in such a way that are econom that makes economical sense in the public Cloud so aliser can you explain to the audience what we have or what we had yesterday and what the vision of what we wanted to take it to sure what we have is is an application that I've called granite and the objective with this application was to look like an Enterprise actual application so it's it's an auto entry system and fulfillment it has a web front end it has a uh mySQL database as the center of it and then it has a series of workers that behave like a bunch of users so or whatever the business function is so in the front of it there's an order entry system where there's an Automation in it because I didn't really have 300 staff to to go and run this thing for me as a production application there's an auto entry uh component that takes a list of our our products and a list of our customers and it orders a whole bunch of product from it uh that goes into the database and then there's a series of processes that analyze these orders and say do we have the stock on hand do we have to order it from a third party supplier do we have to manufacture it ourselves and it goes through those series of transactions eventually fulfilling the order and saying yes we've brought together all the parts we've assembled the thing and and we've shipped it out and then generates an invoice and so this is a distributed application that runs across somewhere between eight and 10 or 12 virtual machines and was designed to run on premises on a a vsphere infrastructure structure so it's a bunch of VMware VMS and fundamentally that led to some expectations about how you'd build your application you've got at this virtual infrastructure there's a very small incremental cost to deploying another VM on it and in order to get that containment of of resources we just deploy another virtual machine another virtual machine for a load Balan another virtual machine uh as an additional web server more worker nodes if we want to generate more workload and so there was this whole collection of of things that are pretty Norm things to do when you're building an application on premises uh including things like using an NFS file share as a location for those um those invoice PDF files to live so that they're accessible across the web servers that make up my um that are behind my low balancer is the web interface yeah so again it doesn't matter why we made the architectural decisions this is the application that we have and this is the application that's actually served us well for the past few years to do all kinds of testing uh and as you're listening or watching this you're imagining well this could have easily been a containerized app it is built on a set of services but it's not and we made design decisions that made sense for our virtualized environment that we had at the time it didn't make any sense for us to run out to roll out a kinetes cluster or even a Docker container cluster all our other applications for VMS we VMS are incremental in cost they're basically free just as long as we're not expanding beyond the capacity of our fairly robust lab environment you know we had two servers with a terabyte of ram so each so that you know we we had plenty of headro there didn't make any sense to complicate our environment by going with a control plane that none of us knew how to manage so that's how we get to where we're at I think it's really important to recognize that also we're a lot of um customers are at a lot of customers have built a lot of buil business processes into these VM based applications that were typically three- tier applications uh and and so they are built with lots of virtual machines and fundamentally that's why we started here is because we wanted to reflect the journey that real production environments are going through and Enterprise organizations yeah and that's the unique thing about the CTO advisor now the signal 65 data center right that we mimic a customers environment we started out with Legacy Hardware we modernized that Hardware we modernized the VMware stack uh we went through this process we connected it to the public Cloud we're able to v motion and expand our vdi infrastructure from the private data center into the public Cloud once you know the everyone was working from home then we could rightsize that again we had a proper or have a proper hybrid infrastructure and we've done some initial work uh workshops and some initial research with Google and uh identifying and migrating these workloads to the public Cloud but without a doubt I think we've all consistently experienced that simply lifting and shifting this stuff like for us to lift and shift this these workloads to the public cloud in VMS would be cost prohibitive like my AWS bill right now is 35 bucks a month and it's 35 bucks a month because the only thing that we have actively running in AWS right now is our S3 source to backup for our uh vaulted backups when we need to do experimentation we use AWS for what most Enterprises use AWS for to exper expent and then we bring that stuff back on Prem however geni and AI in general is different it is very much in the experimental phase in which you want to iterate in the public Cloud figure out what you're going to do what the app is going to look like what the workload uh performance and resources we'll need then we can make uh educated decision on that data that we get from the public cloud and our poc's in the public cloud and either decide to keep it in the public cloud or bring it back on Prim and the one thing that we can't really can't emulate on PR is the speed to value for using some of these AI Services you know uh uh Google Cloud did a really great demo of using Cloud run to leverage uh their AI infrastructure and just make AP API calls to it so if you kind of went over alistair's app we have we can create 300 workers that do imitation order entry Alice I could easily imagine having the agent in a AI environment that replaces that that function that's simulating a something that's a little bit closer to a real customer load yeah I think there also in terms of delivering some business value to what what we're trying to do in this is is using the sales history for a series of of customers to identify what products we're likely to sell and to identify um basically getting ahead of ordering stock for those products because often in in the sort of flows that I set up with with granite is that there's a big latency between the order being placed and the product being produced because we've got to wait for manufacturing we've got to wait for order out well there'd be some mechanisms using AI to do prediction of what those orders are and to improve the just in time delivery of those components um there's then also the the generative AI idea about getting customer specific advertising so as we're starting to bring a new new product how do these correlate and how do we get uh customer specific information out so although the the actual application I built is relatively small the applicability for adding g tools into your application is as much wider the other thing that I think is really important is that the tool set for building a gen application on these public Cloud platforms is far more mature than the tool set for building the same thing on premises and so your cost and time invested to build a relatively simple generative AI Solution on premises is really high because the the tooling is is just here's a pile of you know screwdrivers and Hammers and and bolts uh whereas on the the primary public Cloud platforms here is the actual a a service delivery and this is why we want to be running our gen things on cloud and of course they dependent on having our access to our data that was the important thing about the the demo that of cloud run doing um rag retrieval ored generation that we saw at cloudfield date was that the the knowledge is in our own data and that's what we need the AI to attach to the uh the sort of natural language processing part of the foundation model is awesome but it's locking it to your corporate your internal data and this is why we think we want to move our application from on premises up into the cloud to unlock that data using generative AI yeah so we we can't go from a model in which we have eight always running VMS in the public Cloud no uh that will again be cost prohibitive be at least at least in our use case at least 300 I mean $800 a month with the smallest VMS available in AWS today I think I clock at about $120 $130 a day to run you know really small VMS in the public Cloud $130 a month to to to run those instances so you know we we'd go up to $1,000 a month from you know no no VM bill this this month to $1,000 a month and uh Keith wenton our CFO i' have to explain that to them and I don't I don't want to have to so with that said we see the problem ahead of us we have the app on paper this app is extremely container friendly aliser you've built you know kind of this this this set of apps with services that could easily either be we could either replace some whb Services oneto one like MySQL I can literally just create a MySQL instance in the public cloud and uh use that serverless instance and these runtime s these containers I can run in eks Cloud run uh in whatever as your uh names their thing but first thing we have to do is containerize the app and that's what you've worked on for the past couple of weeks talk to us about that lift sure uh well I did originally write this to be relatively easy to move across into containers so I I knew this was was coming and I didn't want to make too much pain for myself so each of those little processes that create an order um produce a part order a part fulfill these are all just self-contained Python scripts and so getting those into containers was very simple there was the docker file to build those containers is about six or eight lines long there's a few dependencies for some of them generating a PDF I've got to actually pull in the right libraries into that python container so getting the actual business logic code in was relatively easy I did hit a couple of snacks uh one is that I have hardcoded hardcoded the DNS name of the database server into my python code and I'm I'm realizing that you know there is a reason for using environment variables to pass these things into containers and maybe I should be following that best practice design I never said I was a good developer um I also never said that I learned the lessons that I taught so teaching uh AWS training courses about building applications so absolutely it's build your application for the cloud you're using using containerize uh use environment varibles separate out config from U persistent data from execution uh these are things that I'm now having to retrofit partly so the kind of things that are challenging is that I've used a file share for those PDF files and it was just a file share on the one virtual machine that generates the PDF file so it was just running an NFS server and that NFS Mount was then mounted to the two web servers so they got access to the same thing well containers that's not a sensible thing to do um happily at the moment it's just running on a single um a single container host uh and so I can just use a a file system as a volume but I'm going to have to work out what's the right type of volume to present to both that invoice Creator and those web server containers when this gets moved out onto uh onto a distributed environment but the big challenge is that with those virtual machines I had dedicated resources that I had no incremental cost to use within the virtual machine so in each of those worker virtual machines there's a schedule task that runs and it runs in a fairly tight Loop it just checks the time reads the database and says is it time for me to run one of the processes I'm responsible for now that's okay to do inside a virtual machine particularly if you actually go idle in that Loop and then uh release the CPU to be used by other virtual machines and then come back maybe maybe a second later to see if there's something to do but having every container running all of the time in order to run that schedule doesn't seem to be a good idea and so the thing I need to make more native is that scheduling and have the scheduler talk to my container platform rather than just talk to just launching local processors um that's the bit that I'm looking at at the moment my first pass is that kubernetes has a a jobs function and a Chron job function so you can say every minute run this container or run this pod uh however every minute isn't fast enough and so I need to write a scheduler that will operate at the second kind of rate or preferably tenth of a second kind of uh pace and that's the next Hill I got to climb um building that and and if I'm going to do this on kubernetes then it should be an operator and it should be a um container that's designed just to do this and that then integrates with the kubernetes API like kubernetes for that because of course it's the same API whether I deploy on AWS on Azure on uh Google or on premises what I thought was that I built this application a while ago and there's new versions of python and um there's new deprecation of formats and I did have to make some code changes to update for the new version of python although that's a pretty normal kind of thing as you updating python applications so you know the I think we're highlighting a couple of problems when you're going from one architecture type to another always on VM running this is one of those things that I really had a a really difficult time wrapping my head around and and traditional monolithic applications that's running on a VM the computer's running all the time so the cost of having a Damon that was constantly listening was minimal that that was just of course the VM would be on why would it not be on and needing to rejigger my mind to this idea of serverless where one I can schedule something to run or I can uh make it eventbase so I can have an event bus and I can listen out for an event to happen and once that event happened I can pick that event up triggers a container being ran it runs for a period of time but again your your application logic has to lend itself to that type of thinking right and then from an organization perspective you have to think about event buses which a lot of us have we have CFA we have uh well we don't have CFA but a lot of organizations have a CF they have one of the more Legacy uh event bus Solutions out there and they can build around that and but it takes this you know rethinking of the application logic and these dependencies that you just weren't aware of because it ran in the VM that ran all the time yeah and and this is fundamentally the the sort of the center of the idea here is that when you build an application as a developer as an application architect you're building it on a set of assumptions about how the infrastructure works and the on- premises instru uh um assumptions are very different from the oncloud assumptions and lifting from one place to the other rep platforming and redeveloping your application is is a big uplift and to get the most value out of that cloud platform you really do have to start again with a new set of assumptions and that comes back to that whole idea of cost right is is the cost and the time to do this going to actually get us a return When we get this thing into public cloud and we're running this adjacent to whatever other services we want to consume and that fundamentally is the whole point here to get that business benefit and I you know I have two minds of the effort the whether the effort is worth the the juice if the squeeze is worth the juice is one thing is this is known right we we know how to do this this is moving Legacy applications from the private Data Center and the public Cloud this is not rocket science anymore we you know there's no known patterns there solutions to these problems so that lift while labor intensive is a known lift on the flip side standing up AI in a private data center is a unknown lift like uh I had a conversation with Jean Kim last week and he was talking about spending a couple of hours trying to load drivers for a GPU I don't want to pay $1,000 an hour for my developer to try and load drivers that I've I've after you know two or three hours of mucking around with the environment the infrastructure I could have paid a month of experimenting in the public cloud and getting real value but you know kind of a sub factor to those two factors is wow I can get addictive to cloud in both ways in the obviously in the AI way that I'm you know using those run times in a way that I can't easily recreate on PR later on and then two as I'm building services around uh the public Cloud I Hint it to you know you can we can do a mic SQL service and I can consume that I can recreate that on PR that's not too difficult but the service bus part that is where we can get addicted into a certain cloud provider so you know some people might just be asking well why don't you just use the AWS service bus instead of you know uh going through the scheduler problem well now I'm creating a dependency on the public cloud and now if I want to move this application back on PR I have to figure out how to either mimic the API call or I need to find another solution to bring the application back on Prem and these these Cloud Comforts have a way of creeping theirselves into our their selves themselves into our architecture just like the VM Comfort creep its way into our architecture yeah I think one of the things I definitely saw in in AWS since that's what I've specialized in is that it's a developer and ablan platform and it it really does bring a whole bunch of easier ways than than having to go back to the infrastructure team and and get those uh drivers loaded for the uh for the gpus and get them licensed and all of those challenges um and it it goes right through you know having a a message bus having tools like um Lambda so functions as a service or on um Google using Cloud run uh these are all the sorts of tools where you get a lot of value but because you're getting a lot of unique value out of the platform you're going to be tied to that platform you're going to invest development time and there's going to be a cost to switch away so I I sort of flip-flop on this about whether that lockin is a good thing because we're getting lots of unique value or it's a bad thing because we're making the uh cost of switching much higher uh it yeah I think that's a decision above my pay grade so let's talk about you know from your pay grade from your level what has been your experience from two levels one the experience for migrating Granite how much time have you had into this application and containerizing it and then two if we had to get to factory mode if we're doing 120 of these apps what what's your sense of the level of effort when it comes to uh creating a a a modernization factory and people are going ask the question can AI help in this process um so the the first side uh this relatively simple application I got all of the the functions running in containers in about two days of effort and then getting that to translate into running the application on kubernetes is taking a bit more time because of that scheduling and and um getting things to run at the sort of cadence that I want them to run in terms of The Wider con thought about migrating applications from on Prim to to Cloud there's there's definitely a big uplift to doing it well uh we know that lift and shift is not enough you've got to improve your applications either after you've moved them up into the cloud or or before you've got to make them work in the cloud with cloud services natively and so there is a lot of effort in there but there's a lot of patterns in there are that are fairly consistent um and as you say this has been going on for quite a while people have been doing this for years and so there is plenty of prior art to leverage and um generative AI tools AI tools integrated into development environments are very beneficial for this uh it it's simple things like being able to get um on AWS code Whisperer or C genius C genius on on Google um to to make suggestions that's here is I want to achieve how does the codee work so I've never worked with um the with Cloud run and I can get suggestions about how to actually interact with the API for cloud run from this generative AI rather than having to go and read the documentation and that's incredibly beneficial if I'm not going to specialize in this particular API if it's just part of getting moved across part of just onboarding those generative AI tools that are giving me hints it's like having an experienced buddy sitting over your shoulder saying hey here's a better way to do that and without the cost of course of of peer programming uh so yes there's definitely some some value in those generative AI tools uh for assisting developers to to get familiar with new platforms but there is still going to be a lot of work a lot of knowledge of how the application operates how and why it was built this way and those are the things that are going to be a big challenge for Enterprise applications that have been around for the last 20 years where the people who made those decisions May no longer be with us may no longer be in the organization uh this this is going to be tough stuff for large organizations and coming back to one of the other things that we had when we did the project with Google about analyzing what you have on premises and where it might land out on on Google services it's not that you can do everything in isolation it's that in an Enterprise environment everything is dependent on other elements and it's that knowledge of the dependencies it's the Enterprise architecture and application architecture elements that are absolutely going to be uh really tough to to get through and really important to establish very early on uh definitely get some experimentation out there get some handson get Beyond just the intellectual knowledge that it is hard to change things to actually go and G making some changes getting the people who who care and um who are involved in in building these applications to get that hands-on experience because there's nothing like tearing your hair out at not being able to make the right API call to to to teach you where the right API call actually is yeah some people may be looking back like they they really don't understand the Enterprise and why it's so difficult and I think one of the things that you mentioned in passing is a great example you know the NFS server again this is not a production application this is something that you know you threw together over the past few years but in a real Enterprise setting that NFS while the developer is making calls to the foul system he has no he or she has no idea of what the infrastructure and operations team did to make that work like the VM like the it it's a VM yes but that VM might have RDMA storage backing it and that RDMA storage might be exceptionally fast so how do you mimic that in the public cloud is a regular ec2 volume or NFS service in the public cloud going to be enough or are you going to have to subscribe to a higher value or higher throughput service all of these Minor Details uh matter when you're talking about migrating real production Enterprise applications into services this is all data points you need and Architectural decisions you have to make and then there's there's different levels to which you integrate with the native services so we're talking about using a NFS fileshare and moving that from an on- premises NFS server to an NFS service on cloud but maybe I should be changing to an object store particularly for this use case where I'm writing the data once it's not allowed to be modified after the invoice is generated uh but I might be retrieving that data multiple times particularly if I'm using it as an input to my AI system so maybe I should be storing this on object storage but of course that's more work I have to do to make my application work with object storage rather than the the file system storage that I was used to yeah I would love to uh talk to our former colleague Tom Green about this because he does this all day every day at AWS and I I remember talking to them initially that you know they're encouraged to turn the knob as far as they can in modernizing applications and this is the challenge with the Enterprise right is our and I recommend not making too many changes as much as possible matter of fact I would actually recommend doing a v to uh uh uh a lift and shift initially to make sure you get a production app that works and then taking the app apart once you get it into the public Cloud because too many changes too fast and you cannot support your environment I've seen it whether you're talking about data centers to Data Center migrations uh P Tov V top really doesn't matter when your eyes get too big in the Enterprise you find out how fragile your applications are yeah the other element about it is that using Cloud native Services is much easier once you're already in the cloud uh now the flip side to that is that once you've picked up what you have on premises and lifted and shifted into the cloud your cost is going to go up and you're not going to get the benefit of being in the cloud in that state so there's some challenge then around balancing the investment of of keeping the lift and shifted in place and the investment of making it more Cloud native we do see uh Progressive optimization as being a fairly common pattern so we might move our entirely VM based infrastructure up into the cloud and replace the VM based load balancer with a load balancing service and replace the MySQL service with a managed database and so progressively replacing piece by piece and I agree lots of small changes this is the devops way make small changes make reversible changes and very importantly instrument as you go so you identify when a problem occurs what was the change that led to that problem yeah and so with that what's next for us Alis are we what's going to be the next update to the audience what what what would be the next major mouth on for this project there's going to be a couple of different parts uh one is that I'm going to start publishing on my demit test.
c we we we'll still we are still there aliser is at dematas NZ or just dematas demitz on on socials and di. nz for my personal website and I'm ironically still at CTO advisor which is not confusing the name of the business unit is the CTO advisor the name of the company is the futurum group and my Twitter handle is at CTO advisor and Al is also C advisor that's the only thing that's confusing the the the the we did not well I did not think of this naming convention well when I first uh named the company until then again check us out online if you have questions comment below hit us up uh my DMs are open and I'm always available of course via LinkedIn thanks a lot aliser thanks Keith good Tron
