Network as Code Advanced Topics (Video 3 of 3)
Transcript
(light dramatic music) >> All right, you folks are troopers. You've stayed through the first two videos in a series where we introduced the general topic, Network as Code, video two. We talked through implementing network as code, some of just considerations for when you're starting a project. Now we're going to talk about some of the advanced stuff, some of the gotcha's, some of the things that you really need to consider. We spent quite a bit of time on this in the last video, Ned.
And by the way, thank you, Juniper, for sponsoring this series. But Ned, we spent quite a bit of time talking about this near end of the video, and it's a very important topic, network testing and validation. What is that and what are we looking to achieve? >> Right, well, I think there's multiple levels of testing you can do when it comes to deploying your network as code. And it all starts with the initial check-in of that code to whatever your source control process is.
Typically, that's going to kick off some sort of integration pipeline. And it's the job of that pipeline to do some very basic checking of what you've submitted. Is it formatted properly? Is the code syntactically valid and make sense? And maybe you're even testing for some best practices by using static code analysis tools. And we'll touch on specifics later, but the general idea behind a static code analysis is it doesn't try to run the code. It's simply looking at the code itself and it has some rules.
It's basically a rules engine that says, "Oh, you're opening up, you know, port 22 to the entire world through this firewall rule. " So, you know, it might flag that as a not good configuration. So you're going to have some sort of static analysis tool in there. And that's all about vetting the code just to make sure before it even runs that it looks good. >> So, so Ned, as you're talking through this, I can't think of a single product that kind of does all of this.
And as I'm stringing together the test process, I can't help but think, am I introducing another point of failure? Because now there's another human doing with their meat hands stringing together these tests. Isn't that another potential breaking point? >> Of course, you're adding more complexity. That's what we do as engineers, right? We add complexity. I would say the benefit here, and the good news is that when you're building your CI/CD pipelines, you can do that declaratively with code. So it's not necessarily somebody sitting there and stitching all these pieces together by dragging boxes around the screen or anything like that, you can develop workflows and standardize those workflows and publish them for other folks to consume and use.
And actually a lot of the vendor platforms have that process baked into them. So they already have these predefined templates or these predefined workflows that you can take advantage of and then kind of snap in your own tooling and tool sets. So you still are going to have humans involved because you're always going to have humans involved, at least I hope, otherwise we're out of a job. But those humans are going to be doing things that forward your automation goal and they'll be doing it in code, not by manually clicking around in a UI.
>> So what is this called? Is this testing this code? What's the name, is it's just, is this all just part of the subset of infrastructure's code? >> I think so. So we're borrowing a little bit from the concepts that are behind testing software. And so software has a whole bunch of different test types. You have things like unit testing which is just testing that say a function returns the values you expect and errors in the way that you want it to.
So that's at the very essential, I've got a unit and I'm testing it and then you get into integration testing. Okay, how does that function work with the rest of my program as a whole? Not all of these concepts apply one-to-one or map one-to-one to infrastructure and to networking because they are different in the way that they work. So we can try to apply some of these concepts while recognizing that not everything is a perfect mapping from software development to managing infrastructure.
>> So talk to me about integration testing like a continuous motion. When I was building, it wasn't quite a greenfield, but it was greenish. We got a chance to kind of reset a massive network for a Fortune 100. This opportunity doesn't happen often. And we were bringing in the application, it was a mission critical application. It was my job to manage that mission critical application. And they built all of this network redundancy. And I wanted to say, hey, let's take the opportunity to turn off that switch.
Like in theory, like we don't get to do this in the real world in production, right? " That's nirvana, but how do we gradually get there? How do we kind of integrate, merge these pipelines? >> Sure, yeah. So I mean, the testing begins with when code is checked in, right? That's the beginning. And we're going to do some basic tests to make sure that your code is good. But then the next step is how does that integrate with the existing system?
And that can be really difficult to test because you don't necessarily want to apply changes live to your network. So you could potentially have a development instance of some of your network, maybe it's a virtualized version of your network where you can deploy those changes, and then have a series of checks that just validate, hey, the configuration loaded properly on the switch. It didn't barf on any of the commands or the instructions or the configuration that's in there. It's a valid config that will actually load on that switch even though it's a virtualized version of that switch.
And then there's another aspect of integration, which is how does the network function as a whole once you've deployed your updates? And that can be very, very difficult to test in a non-production environment. So there are certainly tools out there that will attempt to make a digital twin sort of of your existing environment, apply the changes there and review the results. But ultimately you're always testing in production, right? Eventually it has to hit your production network and you're essentially testing there.
So what's really, really critical in this whole process is having a complete feedback loop to capture what's happening in the production environment and have that inform your development process for the next iteration of your code. >> So let's move on to another topic. Monitoring. How does network as code impact monitoring and analytics? Seems like there's opportunities there but what are some of the advanced things we can start thinking about once we move to network as code? >> Well, certainly assuming that you're monitoring an software packages and devices support it, you can deploy them with network as code.
>> It seems like we, there's a theme I'm developing here, I'm hearing. >> A friend of mine likes to call it everything is code, right? If it has an API endpoint and you can program against it, you should be defining its configuration using code as much as possible. And I think if there are any SREs watching, you know exactly what I'm talking about. That's the nirvana of the SRE is to automate all the things that can be automated so you can move on, do something else.
So that's a portion of it is just setting up that initial monitoring and analytics. It's something that some people forget to do when they set up the switch or they set up the router, they forget to turn on proper monitoring or get it integrated with that monitoring package that you have somewhere else. Oh, I set up a new switch, but I forgot to send the ticket to the monitoring team to add it to their list of network devices, that sort of thing. When you have that monitoring portion defined using network as code, you don't have to remember because now you've created an integration where a new network device is added, it automatically gets integrated into your existing analytics and monitoring packages.
So it's that dynamic discovery and integration. So that's certainly a portion of it. But I think another important portion of it is the ability to capture the impact of your changes. And that kind of gets back to what we were just talking about like I deployed my code, did I break anything? That's certainly important. But another and possibly equally important part is what were the actual impact of my changes and did I achieve the goal of those changes to begin with?
'Cause we don't just change the network for funsies, right? It's not, Friday we're going to deploy some new network as code and then go out and have happy hour. You have a business reason or a technology reason to deploy changes to the network. And so defining what the point of the change is and then figuring out how to measure the impact of that change to make sure that the change you made actually is reflected in the performance. That's the job of monitoring analytics.
Oh hey, you made this update to the network and now customer requests are coming in 50% faster than they were before because you streamline something in the network, that's awesome. You get to report that back to your boss. I improve the network performance so that you're getting more customer orders per second. Fantastic. >> So as I think of the CI/CD process and things that we wish we could do, but we didn't have the people or processes to do it, it was too expensive to do or too burdensome to do it every time.
You know, we could create CI/CD processes or pipelines that would kick off specific monitoring for a specific amount of time off a set of ports, let's say port mirroring, generally speaking, is expensive from a resource perspective, but after a certain change we want to always mirror a port for let's say two hours. So we collect that data and if there's another trigger from the monitoring tool that says, "Hey if we reach this threshold, take this action," and this action may not be disruptive like making configuration changes, it could be monitor this other thing that is, kind of this limited resource that we can now put on to collect more data and make better informed decisions.
>> Yeah, absolutely. And at this point, I won't say like CPU time is cheap, but it's a lot cheaper than it used to be. >> Right. >> Storage isn't cheap, but it's a lot cheaper than it used to be. So the ability to capture all of this information is certainly there. The other big challenge is then, okay, I got all this additional info, how do I analyze it? How do I munge useful information out of it? And so that's not really a network as code challenge but it's something that's going to feed back into the loop of your development of network as code is having some sort of data analysis tool that can give you useful insights into the information that you're gathering.
>> So let's talk about our last topic in this series. I think one of, if you're a networking person, you've dealt with both sides of this, implementing your security policy via the network and then ensuring, just proving that to some internal or external audience. So let's talk about implementing security policies through code. I talked to a bunch of folks about security is code. That's a thing. Where do we start with our security policies through code? >> Sure. So there's a whole bunch of different policy engines out there that will analyze code, compare it to some set of policies, and then give you the results.
One of the most popular ones that I've been working with for a little while is called Open Policy Agent or OPA. And that has the capability to analyze anything that is expressed in JSON and compare it to some rule sets that you've defined and then give you results based off those rule sets. And what can you express with JSON? Well, just about anything. So whether that's doing analysis of static code analysis, so just what does the code look like? Or it could be, I have a plan set of changes that I want to apply to my network and I can look through the plan set of changes and make determinations of whether or not I find that it's secure all the way up to analyzing the actual running configuration on network devices or servers as long as it can be expressed through JSON.
OPA can take a look at that and make some policy decisions, say, oh, well someone went into this switch after the fact and altered something and it's no longer in compliance. And that compliance can be defined usually through the security and compliance teams in your organization. They set the policies and then they allow you to test for whether or not you're in compliance with those policies. >> And one of the things that frustrated me to no end when I did network administration and operations day-to-day in large organizations is when the dreaded auditor comes in.
And I think I wanted to talk about two topics within this. >> Sure. >> One, how do I answer the requests from auditors when I'm living in an infrastructure's code and networking code environment? Because I'm not in my mind going back to the individual switches, pulling configs, going to backups, et cetera, to answer the requests from the auditors. And then the second one is, how do I help the auditors trust those artifacts I'm giving to them as proof? So let's do the first one first.
Like how am I pulling the request? A sample request is show me that authentication is configured on every network device. >> Sure, yeah. And I mean, that's a request, pretty common request that comes in. Now let's assume that you've defined in your network as code authentication policies for every single network device. All you need to do is run a drift detection essentially against all your existing network devices. And that gets back to the get set and test that we talked about in the previous video.
You're just basically running the get and test portions of that. Get me the configuration from every network switch, test it against my defined configuration. Is there a difference? No, there's not. Awesome. And hopefully the answer is no, there is not. And so you can go to the auditor and say, here's the run that I did against all my network devices, it found no differences and here's the configuration that I've defined in code that clearly has the authentication policy enabled.
There you go. I'm done. I don't have to tap every single switch myself and pull the config and dump it out into this giant document that I deliver to them. Here's the runner that I went through that tested it against all the switches and then here's the actual configuration it was testing against. You're good to go. >> So the smart auditor will come and say, well, there's a whole another control plane. Let's authenticate that the folks making the changes because we're no longer making switch level changes.
We're not going into the switch to configure changes. This whole other team is doing this platform team. How do we ensure who has rights to make changes, if this quote unquote "system" is making changes? >> Right, I mean, because it depends on how you've secured the workflow. So a fairly typical process is everything goes through code. You're following sort of a GitOps process. So the way that I make changes in a system is that I submit my changes via code to the repository and that kicks off via a webhook, some CI/CD pipeline.
And in that pipeline will be an approvals process for the changes. And so someone, whether it's an automated process or a manual process, needs to vet those changes, determine whether or not the changes should be allowed and then approve those changes. And what you have in the repository is a record of exactly who committed the code and when they committed it. And in your pipeline you have a record of exactly who approved that code and when they approved it. And so you can trace the full change of your environment through that entire process.
And that doesn't mean that sometimes you don't need to break glass, in the case of a hard down situation where you need to make immediate changes but that is hopefully an infrequent event and that you have a well-defined process for getting approval to break glass and make changes. >> So this all starts with, you can't automate, you can't code processes that don't exist. Well, at the end of the day, the system is there. All these systems that we've talked about in this series, those systems are there to automate or codify the things we've already written down on paper, the processes that we've already talked about, the operational issues we've controlled for.
I've seen CI/CD processes break entire systems because people didn't sit down and write down their existing processes and then build a CI/CD pipeline that supported their existing pipelines, they tried to recreate the wheel and break literally 30 years of integration test processes, et cetera, without really thinking through. And I think, Ned, that summarizes the whole series. What we're trying to do is scale our operations in a way that meets the bell of today. The CTO advisor's premise is that hybrid infrastructure is here to stay.
We cannot afford to have a bespoke approach to any infrastructure, whether that's network storage, compute or public cloud. We have to have processes that scale, take humans out so we can put our people on smarter and harder problems such as network to cloud networking, cloud to cloud networking, cloud to cloud security. These are problems we need to rededicate our staffs to solving. Ned, any less comments for our audience? >> I think you hit the nail on the head there. It really is a matter of automating existing processes but most importantly, you don't have to twist the tool, you don't have to twist yourself out of shape to fit the tool.
All these different tools that exist are extensible and customizable. And so you should customize the workflow and select the tool that meets the existing shape and workflow of your organization. com. com. Ned, where can folks find you? com. All of my links and other content are all hosted there. >> All right, until then, we'll talk to you next video series.
