Platform Teams and Super Cloud with Rob Hirshfeld
Transcript
sh is calling the AWS CLI but it's doing it inside of digital rebar and one of the nice things is because it creates a work order for that it creates a job for that it creates a task it's audited you could actually start adding different parameters and checks you could say you know what you're not allowed to do anything but the ec2 CLI here or you're not allowed to run these extra commands or if you run certain commands I want to set a log or an alert or generate additional events so all those things become possible around what would ultimately just be an exact syntax substitute for the AWS CLI so people may be asking like okay I don't grock the value of this ah let's let's let me kind of tell you why I asked for it I'm not a user of digital rebar I'm abstracting the way why I want this why I want this because I think from a managed service provider lens if I'm a managed service provider and I'm servicing multiple customers AWS environments and I get to the point where I have leverage I have some scale and I'm getting some cost redundancies I can now do one of two things I can proxy all of those requests to my same AWS account and get the advantages of the reduced pricing without changing the interface for my individual customers each one has their contacts within in this platform in this case digital rebar each one has their contacts and then I can continue that abstraction into my managed service provider offering or I can do something a little bit different I can make a API compatible Solution on premises and now instead of servicing the request via AWS resources I can if I have some gen 7 gen 8 Hardware that's going to do the job and this is all you know basically free to me equipment why not and the from a developer experience it's a it's a one-to-one experience is is that thinking valid it you know there's there's definitely the you know each cloud and every every system has unique syntaxes so so it's very hard to make a universal AWS to something else translator but what what this lets you do is you can actually start looking at what the calls people are actually making are and then saying oh wait a second there they need these actions let me provide those actions so there's an audit capability and there's also a there is an opportunity to rewrite the task and then and redirect the task but you have to have an API abstraction point to capture that so what what you're describing to me is a very real use case of I'm running these AWS CLI and we're using this as a proxy it's nice and concrete but I'm running a action inside of a broader workflow I want to be able to audit it check it redirect it if I need to conform its security one of the things that I get excited about is I could actually do a Dev test prod type of work where the script we're using could actually be behind the scenes redirected to Dev instances or prod instances right you could actually have the same script now used in different cases in multiple cases and the infrastructure is code repeatability of your infrastructure starts going way way up all of these are are important uh interaction points that are not easy to replicate from just the AWS CLI right yeah or anything right it's and these are a great Insight so let's you know wrapped away even from the digital rebar use case I love this listening mode you know we look at the 238 services that are in AWS the reality is what the average organization might use 10 if not that and of those 10 when you drill down and say okay I want to recreate some of these services or uh a subset of it well let's look at ec2 you know of ec2 am I using the different I don't know is it I don't know if it's a with the sizes with a combination of size and type if there's more I think there's more than 100 different size and type combinations which actually I think I think they crossed over to a thousand just of instance types so naughty just flavors of instance types do I need to recreate every form of that instance type can I just proxy that these that basically we use 20 different insta uh ec2 types and I can recreate those ec2 types on-prem via my on-prem hypervisor of choice it doesn't really matter I can look to see okay developers in their development environments almost always go to these three different types I can do that on-prem and not recreate a bill when they push it to production they use these eight different types we're going to continue to push those to production on AWS but these other ones we're going to service on-prem that is kind of the goal but as me and you discussed last week there are unique problems uh because you get into even in that limited set you get into edge cases you do and and it one of the things that I think is interesting as a platform team or an operations team is coming in you know where you can you know it's where and how you can support people as they're coming into these systems is actually a really important component to consider so there's edge cases in how things get mapped between different infrastructures right you can't just take um an AWS call and automatically map it into an on-premises call a bare metal machine or a VMware machine VM those those mappings do require a certain amount of understanding and how they work um but one of the things that I find is just how do you answer the phone when somebody says they have a problem and can they replicate the can they replicate the challenge right one of the things that's interesting about edge cases is that a lot of times an edge case ends up being something where you need real expertise or support to help resolve what that problem is and so part of the the value in this is you actually want to be able to get your experts on the line have them duplicate exactly what happens see the logging figure out what what went into that circumstance and then ideally replicate it and even more ideally provide guard rails like you've been describing for the next pass on this because right it could be really easy to write a series of commands that can't execute because somebody doesn't have the right permissions they're not in the right VPC they don't have right the Omni the Amis Amis that they're using for the machines don't exist in the regions that they're trying to execute them on right there's huge numbers of failure possibilities and how all this stuff gets gets fit together and I think one of if you're listening to this and you're thinking through wow I could proxy between Cloud providers we one of the terms we use in our conversation is supercloud this is the beginnings of creating a super class developers all develop in AWS CLI or terraform pick your preferred language to describe your infrastructure's code environment you settle on one form and then that form can be reapplied across multiple Cloud providers on the platform group sorry you can now proxy and say oh I can intercept a request that was going for AWS and now I can send that request to Azure that was how I was thinking from the architecture and drawing this out perspective but we got into some of the details about that the talk to me what happens when I request the AWS machine from Azure and and we proxy that request you know it's it's interesting because at some level machines are machines or machines um which is which is you know thankful I'm thankful that that when you're running a Linux machine you know of of a certain flavor and Os distribution um that OS distribution usually translates pretty well Cloud to Cloud you know on-premises off-premises bare metal Edge wherever you are there's certain certain components that you can take for granted once you get into a certain OS unfortunately we're still in a very heterogeneous world where if you're using Amazon's Linux that's different than Oracle Linux then Centos Linux then uh Ubuntu Linux right you you know you can't get install can't even app get installed across all the lit all the uh Debian distros anymore um so we do have some real we do have some real differences um you know it and this is this is the challenge I actually like to describe this as a fruitcake instead of a layer cake people want people want uh nice clean layers like oh the storage layer is different it's the same in this the OS layer is the same and all that but it doesn't actually work like that like how a disc gets set up or a network gets set up actually has to be accounted for throughout the layers above that um you actually have to account for differences between the systems but but to your point if I can say if somebody says give me a uh Alma Linux machine of a certain generation then I can get that from pretty much anywhere put an SSH key on it and give it back to you um with and you can say I want this much RAM and this much storage and you can abstract that successfully across pretty much every platform that you that that we encounter which is every every environment um the challenge becomes and now we're straight back to our proxy case the way you ask for that and the the syntax you use to build that generic unit is highly variable across the across the clouds even within something as simple as machine ID if you inventory off of machine ID if you create some type of sock uh uh pipeline based off of the machine IDs you've you've kind of figured out the identifiers within the AWS machine IDs and you build some process on that well an Azure machine ID is going to look very different than uh uh a AWS machine ID which looks different than a g and then the gcp machine ID and even how you figure even how you figure out which which machine ID you have is variant across I was just about to say because the machine ID field may not even be the same field across Cloud providers so that metadata problem becomes a operational issue because these unique identifiers just operationally we us op guys we optimize based on the on the things we can optimize on and machine ID ironically becomes one of those things that become critical unintendedly yes so Rob we we talk about this stuff all the time I was just talking to a young CTO and he was saying that he just doesn't have relationships and I'm thinking man rob used to run this this weekly thing called uh uh uh Cloud 2030.
we're surprisingly getting a little close to that man you need to push out 2014. uh if people want to engage with they want to like bounce ideas off of you how do they find you how do they engage with you we are still having a weekly devops lunch and learn round table and it really is a round table people bring topics and we discuss it um and I love the format because we spend 15 minutes completely off agenda and then um usually we'll try and jump back to some some planned topic of the day which is you know some kubernetes or platform engineering or API design classic devops stuff people can find out more about it at dot the 2030.
com uh you can at me on Twitter at CTO advisor I'm just looking at uh Rob's uh feed out his window and looks like he's getting the package so Rob will let you go talk to you next episode of the CTO advisor Keith thank you it's been a pleasure
