Patterns and Practices for Infrastructure as Code - Rosemary Wang

Machine-generated from the episode audio and not hand-corrected, so names and technical terms may be imperfect. The audio is authoritative.

Hey, you're listening to the CTO Advisor podcast. We're continuing our theme of talking to presenters from the CTO Advisor virtual conference held April 2020. The height of the pandemic, I'm going to call that the height of the pandemic, two months after it started. This episode, we have Rosemary Wayne, Developer Advocate at HashiCorp. Rosemary presented on infrastructures code and scaling infrastructures code. Her role at HashiCorp has changed a little bit. Before she focused on Terraform, now she's, what was it, a little bit of console, a little bit of everything, right?

Yeah, a little bit of console, a little bit of vault, and a little bit of boundary. So anything security related, somehow I ended up getting some security knowledge that I didn't intend to get. So actually, it's ironic. I must have had, like, in the back of my mind, talking to you right before I did my CTO short. At this point, everyone should be familiar with my CTO short, it's longer than a tweet, shorter than a blog post, kind of my everyday dumps of just brain dumps.

And the brain dump that I had was, actually had HashiCorp in mind. It was the ideal of connecting or a Lambda function to a legacy Oracle DB in the security that's needed. Like on-prem, we do that much differently than how we do it in the public cloud. Actually, I'm not sure how we do it in the public cloud, but you're writing a whole book on patterns around security, et cetera. Give us an overview of, like, what the book will be about.

Yeah, so I discovered that when I wrote Infrastructure as Code, it wasn't enough to learn the tools and it wasn't enough to learn software development patterns. There was a little bit of a gap in understanding of how to write Infrastructure as Code and work in these public cloud or cloud paradigms and doing it securely and making sure you could collaborate on it. So something that I talked about at the CTO Advisor Virtual Conference was that the problem isn't necessarily about enabling on the tools.

It's about enabling the tools across your team and across your organization and making sure you can do this no matter the underlying platform, no matter the cloud provider, no matter if it's data center or cloud. And so many of these gaps I wanted to address somewhere. They were in my head. They were in different places. And I was like, I need to consolidate all of them. So I really took a look at what are the patterns you need in order to understand how to secure your infrastructure as code and what are the things that you need to understand from a compliance standpoint, whether it be cost, organizational policy, or even just ways of working, collaborating with other people and making sure you do it securely as well.

So it's interesting that you said cost because cost is one of these things that are, you know, starting to get an equal billing almost with security. But let's stay focused on this security conversation because I like where you're going with this. Help me, you know, connect the first dot. The first dot, there's policy, like what is our security posture and what's our policy around our infrastructure, whatever that is. Help me to think in the correct context before I start to convert that into some technical into some type of technical implementation or of that policy.

Yeah. I see what you're doing there with billing, the little pun. So with security and policy, I think that there are two sides of in some ways the same topic because depending on your industry, depending on your environment, depending on your topology, you will have a different kind of security posture. S. healthcare, you'll have a different kind of security approach than perhaps someone who is in e-commerce and is responsible for personal information, right, personal credit card information versus healthcare information. And so we not only have the security standards set by best practices of our industry, we also have security standards that are set by cloud providers.

So for example, in the case of connecting to your database and your Oracle database and your data center to cloud, right, that's probably over some kind of private networking that you've connected, whether it be VPN or otherwise. You probably don't want to put that over a public network. That makes people a little bit scared to do it and you want to make sure it's, yeah, exactly. So from a technical implementation view, you're, you know, there are a lot of these check boxes, these configurations that you need to tick off, and it depends on your industry, it depends on your topology, it depends on what you're doing.

But at the end of the day, there are some things we all agree on from a security policy perspective, depending on the cloud you're on, and there are some things that you end up maybe taking it to the extreme if you want to be even more secure than you did before. And that's where zero trust security, the buzzword, so to speak, starts to come in. So help me decipher between like a pattern and a policy. I know a policy is something like, oh, we will encrypt data in transit or data in rest is a policy.

Translate that to a pattern, especially when it comes to infrastructure as code. Yeah, so policy is something that you are expected to conform to. A pattern is something that you might decide to apply for a long-term gain. So we talk, the reason why I considered something like an infrastructure as code pattern in the first place was because when I had, during my short stint as a not expert software developer, I learned a lot about software development patterns. And it was never told to me that you're required to use these patterns, that these patterns might make your life easier.

You get a long-term benefit, whether it be maintainability of your code and sustainability in effort across your team when you apply these patterns. And so policy is something that if you don't have, there are likely some legal, financial, and occasionally technical consequences to not having that policy. But a pattern is something is, well, you could get away with not doing it. It's optional in some ways, but it will really help you from at least an operational view. So let's talk about a legacy approach to patterns.

One of the things that I would, in our example of the Oracle database connecting to the, or the function connecting to the Oracle database, in legacy infrastructure terms, we have an application zone, server zone, and then we have a database server zone, and that is part of the pattern. I don't have to do that, but that's good, that's a good pattern. Can I implement a technology such as a firewall and have, and say, okay, this server can talk to this other server across this port?

That is a pattern that I can apply typically across industries, across policies, et cetera, correct? That is correct. So what would be a infrastructure as a code equivalent to that pattern? So it actually has to do more with implementation, and that's what's really powerful. In infrastructure as code, I'm declaring that it's not just functional, that the serverless function is connecting to the database. It's that I'm doing it over a private network. I'm saying declaratively, I'm going to do it on this private network.

I'm going to do this encrypted in transit. I'm going to make sure that I use a, I'm going to use a database secret that I've retrieved from either secrets manager or doing it in the most secure way possible. And so part of the trick is understanding how to use infrastructure as code to better assess policy early, right? Because the problem that we've had with policy and infrastructure in general is that we think about it after the fact. We implement something and we have to think about it and ask our security teams or compliance teams, is this right?

Does this seem okay to you? And then when you find out that you're not supposed to put the serverless function on a public network and you're not securely connecting to your Oracle database, that means you have to go back and you have to fix it somehow. And so the idea behind using infrastructure as code with policy means that you can, what we call shifting left, shifting the security testing left. So you're able to assess your infrastructure as code and apply some patterns that can sustainably implement and construct the infrastructure.

But those patterns will help you assess whether or not that policy is being, is actually being conformed to. So this hark is back to a podcast I recorded, actually I recorded as we're recording this, I recorded it earlier and today it's going to publish either before or after this podcast I'm recording with you, but with Martez Reed and we talked about abstractions and how powerful abstractions are for building scalable systems and speeding up things. However, abstractions also can help us to kind of get in a lot of trouble.

And one of the things that I immediately think about in our conversation is, you know, Kubernetes is a powerful platform for abstractions. So I can create a YAML or a set of instructions or intent that says, hey, transfer this data from site A to site B. But if I don't use the pattern that says that I need to, that it needs to be over a secure network, that the data needs to be encrypted in transit, that it needs to be encrypted at rest, if I don't clarify that at the beginning, then I may end up inadvertently exposing my organization.

So this is the gap between like developers and what developers do, the security group and operations that developers may not necessarily know what patterns they need to implement. And we need to help them with like pre-established patterns or templates. How do we kind of solve that, solve for that practical cultural problem that developers don't know what they don't know? And it's easy. Everybody doesn't know what they don't know, you know, I think as even an operator, sometimes I would be like, oh, I didn't even realize the security, the security configuration needed to be in existence, right?

And I think the funny thing about abstractions is that it's easy to, it means it's easy to use, but you lose a little, you lose sight of the technicalities and the nuances underneath it all. And so that's why there's a balance between, I think, abstraction and offering, offering a tool as is, or offering the, the configuration as is. And I think it's self-service, that's the ultimate definition, right? Can someone self-serve and empower themselves to learn the security or policy that you have in place?

The problem is that a lot of this is an, what I like to joke as an unknown known. It's something that everyone, someone knows in the organization. Someone knows in the organization. Someone knows it, but we don't really know that we, you know, that we know, we don't know it or something. So part of this is shifting the unknown knowns into something that is known by everybody. And the ability to self-serve and empower yourself to learn some of these things actually is a much more scalable way of enabling many different teams, whether it be security, platform engineering, infrastructure engineering, network engineering, developers.

I think that the, one of the, one of the things that changed me in terms of understanding this idea of self-service is that I, let's say I wanted to learn data, how to, you know, how to create data pipelines in the cloud. I was able to look at infrastructure as code, effectively code as documentation, understand that, hey, I could use it really quickly, but if something went wrong, I could dive into it and learn what was going on and understand some of the technicalities.

S. centric, who knew? This is basic security. It's education. Like, I can do all of the technology in the world, but if a user gives their son or daughter their password and token to connect to the VPN, eh, you know, security just went out the door. So that's the base level of education. Then there's the enablement. You have to make security relatively easy to implement. And we can harken this to, you know, something as simple as immutable backups.

If the, if you have a laptop and you have end users that are now having corporate data outside of your organization, you have to give them basic tools to be able to share data, but also encrypt data without it getting in the way of their day-to-day. So for developers, and we're thinking about infrastructure as code, what are some of the things as operators and security professionals, what are some of the things that we can surface up for this self-service capability so that developers can be best served?

So even if they do know that they can implement, that they need to implement these techniques or these patterns, that they are actually able to do it, because even if you know, you need to do a secure connection, you have no, no, no idea of how to establish a VPN. How do we make these things easier for them? Some of these things are, it's going to be the most difficult thing that anybody will do. I can tell you that there are people who build their own abstraction layers around infrastructure and all of these patterns so that development teams can just do it out of the box in sort of a, in sort of developer-friendly language.

So this is where we get service mesh, for example. And when we, that's a whole other discussion, but you know, so one of the things that I focus on is creating secure by default configurations as much as possible, right? So security is often something that hampers functionality, right? You implement some security measure and it breaks something. So making sure that things are secure by default, that you invest early on into testing and putting secure configurations in some consumable infrastructure modules, so someone can self-service that.

It's very powerful, right? Because no one wants to sit there and dig through the most secure configuration. They want to know this is the secure configuration. Right. Yeah. Because it takes time. It does take time. Yeah. It does take time. The second thing to think about is, you know, understanding patterns of development of even development and applications, right? So in, I guess, in the Kubernetes space, but also with the advent of containers, there are bigger questions around, do we really have to make changes to application code to do some kind of security function?

Good example of this is getting secrets dynamically, right? Do I have to change application code to call out to a secrets manager and get that information? Or can I run a process that isn't in the application code at all? Retrieve the secrets for me, retrieve the credentials for me, and I just, you know, grab it from a file somewhere, right? As the application. And so these questions make it much easier. Yeah. I like my secrets like I like my water and clear text.

Yes. We all like our secrets. And for those of you who, I mean, I've dealt with secrets and this idea of how do you get a password? How do you store, essentially store passwords and retrieve them as needed? Back in the day, we just put them in the batch file and trust that no one ever connected to the server who didn't need to read the batch file. But it's not that simple anymore and not just malware, but everything is so much smarter and so much simpler to consume.

Like we're thinking about these concepts of how they make it easy for developers to consume and deploy security. The bad guys are making it extremely easy to consume and use these toolkits to take advantage of when we don't use these patterns. So Rosemary, speaking of patterns, when will the book be out and how can people kind of find you and your research? Oh, thanks for asking. So the book is available on early access through Manning Publications. It's currently titled Essential Infrastructure as Code.

But if you search by my author name, you'll be able to do the, you'll be able to retrieve the early access digital copy, hopefully with a publication date of next year. There's a number of chapters out, so stay tuned, follow along. And if anybody ever has feedback, they can always reach out to me. So I'll put a link to that in the show notes. com. Rate the podcast in iTunes, it helps, or in your favorite podcatcher. You want to talk to me about putting my secrets in a proper repository, you can DM me on Twitter at ctoadvisor, DMs are open, talk to you next CTO Advisor podcast.