Replay-GitBOM. It’s not Git or SBOM

Machine-generated from the episode audio and not hand-corrected, so names and technical terms may be imperfect. The audio is authoritative.

All right, another fun CTO advisor podcast. We have Ed Warnke. You can go back and look at a podcast that knows from a couple of weeks about Ed. That's not important. What's important, we have Ava, who is a non-binary hacker working in the Azure office of the CTO. Welcome both to the show. Ava, welcome to the show for the first time. Thanks so much for having me. Ed, welcome back. Good to be back. You know, you get less love.

You're not loved less, but you get less love the second or third time you're on the podcast. We're all, we're friends now. So at the end of the last conversation that I had with Ed, we talked about this GitBomb project and it piqued my curiosity and said, well, if we're going to have the conversation, we have to have Ava on to have that conversation. So Ava, tell us, what is GitBomb? GitBomb, it's not Git and it's not an S-bomb, but it builds on both of those concepts.

So everyone is probably familiar with Git, the version control system, but underneath that, Git uses a certain type of data storage structure, a Merkle tree or a hash tree. And so we're using that. And a lot of folks have been hearing about S-bombs or a software bill of materials. As part of the executive order last year on securing our software supply chains. And so we're working in parallel to a lot of the work other communities are doing with S-bombs to build a, can think of it like an artifact tree of fractal depth.

You want to go all the way back for any binary you've got to be able to see the hash tree that describes all the artifacts back to every source code file, every library and every intermediate step along the way. And do that in a really compact, concise way that's compatible across different languages, different ecosystems, different build tools. Because today, especially with open source, it's all heterogeneous. There's a bunch of stuff that this enables, that's sort of the under the hood description of it.

So I get the need for software bill of material, the idea that I need to know where my code came from and who contributed to it, or at least what identities contributed to it. And I get the idea of Git, the ability to have a repository for version control as we're making the two. They seem adjacent, but in my mind, without giving it some thought, there seems to be not just overlap, but kind of some conflict in the overall objectives of each. Like they, I don't know if they necessarily in my mind go together right away.

What's the goal in taking Git underlay in SBOM? So SBOMs, you picked on a couple of things that SBOMs do really well. It's the provenance, who built it, where did it come from? What's the software license? All of that metadata we set aside, that's the preview of SBOMs. We're not touching that. We're looking just at the artifact tree. What code, which exact file version, not just like version, string, number, but what's the actual hash, the fingerprint of the open SSL version that was built into the library, compiled into the package, built into your Docker image?

h files that have a known bug in them. And you don't have the source code or the Docker image, but if you have the GitBOM tree, the hash tree, you can immediately tell if the vulnerability might be there. So- Okay, go ahead, Ed. Yeah, so the precision also matters because a lot of times when people are looking at these things, they think about them at the level of packages. And a package will often have a whole bunch of stuff in it, which may or may not have been stuff that you built.

So one example from long, long ago, and the Postgres folks have since cleaned this up, is PostgresQL in the early 2000s, fantastic database, huge fan, BSD licensed, and very, very, very well done. But they had a contrib directory in their repo. And in that contrib directory was a whole bunch of stuff with wildly varying licensing, with wildly varying degrees of quality and security to it. And so you couldn't actually know if I just told you, okay, that's the commit in the repo that I pulled from, or that's the version of PostgresQL that I'm using.

I couldn't know whether you were really secure or wildly insecure, because it depended on what got built into the system. And so we're basically taking this tree of ingredients to the level of precision that we actually know specifically which source code files got built into the system that you're using, which shared objects you're linking in, which pieces all the way up to the running executive, in a way that is completely agnostic across languages. So C, Java, Go, any language that you would actually care to work with, we can actually capture that information with fidelity.

So if I use the analogy of an ingredient list on a can of soup, the typical SBOM layer is, well, it has corn, and the corn came from this factory. Maybe it has that, maybe not. The GitBOM layer is, well, this is the specific farm at a specific time that it was brought in, which is much more useful if you're trying to take a bunch of cans of soup off the shelf because there's something wrong with them. You don't wanna just know what has corn in it, but which cans of soup have specific origins of that ingredient.

A bunch of questions. I can see how this would be once we get there, and then we'll talk about maturity, like, are we there? Once we get there, there's all kinds of usefulness for it. So back early in the late 90s, early 2000s, when we were dealing with Y2K, so literally late 90s, in 2000, the hello, we've all, for us on this line, we've dealt with this, which is, I had to do desktop remediation. So for thousands of desktops, and version control was insanely difficult.

It is a very difficult problem. What version of, not only what version of Word do you have running, but what the dependency library for Word changes every time because of the way that Windows packaged, because the way Microsoft packages applications in Windows. That can change all the time. So not only are we looking at what is the version, the major or minor version of Word, but have the dependencies changed? So is the overall thing Y2K compliant? Because the thing is not just the version number of Word.

So fast forward to now, we're worried about open source code security, et cetera. We have not just packaging problems, but this whole problem, how do we do this traceability? So my first question is, like, where are we at? Like, is this something that enterprises can say, awesome. Now, every open source project that I've ever used, I don't have to roll out a couple of dozen people to go out and trace back SBOM for all of the audit, all of the open source project in my library or my portfolio.

I can just contribute and consume GitBOM and we're done because that's what I would like to do. I would love that too, but we're not there yet. It's pretty early days for GitBOM. I'll say we've got several proof of concepts right now. So folks can certainly come kick the tires and try it out. We're working on getting adoption into different tool chains. And there are a couple other changes in the ecosystem that are highly supportive, if not necessary, at least really help, like the reproducible builds effort in Debian and a couple other languages.

Also really, they go hand in hand with what we're doing. So I love the actual idea and the concept and the work ahead. A couple of technical issues or questions, not issues. From a early build perspective, where are we at in like just agreeing on things like schema, et cetera? So we were actually just describing some of the, discussing some of this in the GitBOM community. So GitBOM started with what I would refer to as the existence proof version of the idea, which is you have something that has enough meat on the bones that people can start building tools to beat on it.

And where you can start having conversations with the broader ecosystem, because again, we're trying to keep, to do something that actually works across all of the languages, packaging formats, et cetera, all the way down from individual source code files as the leaf files in the tree, and all the way up to running systems where you're even beyond the level of just, is my executable secure? But taking, for example, an executable that pulls in a shared library, is the running executable secure? So we had an existence proof level idea for this at the beginning.

We've got two parallel tracks in operation currently. Track one is spec, which is a bunch of folks hammering out the very specific details that can be handed off as a very simple set of things for build tools to implement. Because a big part of the point of GitBomb is to understand that the authoritative entity for a build step is the tool that performs it. So for example, if I've got something compiling source code to an object file, the compiler is the only entity that authoritatively knows what the inputs were.

And so that's the entity that should actually be imputing the portion of the GitBomb from that step and sticking an identifier in the output object that references that portion of the GitBomb as a Merkle tree. Quick question before, because I had this early on before I lose it again. Immutability of that token, of that cert. So the compiler compiles it. Okay, the compiler has a cert, I know the compiler and its identity is validated. And then the object that I write the executable to, how do I ensure the immutability of the object after it's been written?

So this is where GitBomb is actually an intensely non-greedy as a concept. So we're trying to solve one very small problem that is broadly applicable, which is how do you construct and convey the artifacts trees with extremely high fidelity? You've brought in a related problem that lots of other people are working on, which is how do I sign an output artifact from the system and guarantee the immutability of that output artifact? There are lots of, there's lots of effort that has gone into this historically and that is currently going on with this with things like SyncStore.

And our general attitude is to compliment such things rather than trying to compete with them. So how that artifact gets signed is, it's certainly a thing you want to have happen, but GitBomb itself doesn't sign the artifact. What we do is construct the artifacts tree. So if you're given an artifact, which has been signed, it will have an identifier that points to a Merkle tree, which is thus immutable. And so if someone, you can now know what went into it. So immutability up and down a stack.

So you want the tree you guys are working or you folks are working on the tree itself and making sure that that's immutable. And then it's just pointing to a reference outside of your domain of control to say, okay, if we have a central repository, a central tree for almost every project or every project imaginable, that we have a standard on software development, how to sign these detailed elements that you're working on from a scheme perspective in the tree and then outside of that, whatever output.

So the input into that is signed, the output into that is signed, but the tree itself, you own. If you think about the broad domain of supply chain security, there's a lot of different problems, in the air quotes there, of the people are working on solving right now. And I got involved in GitBomb last year because I didn't see anyone working on this problems on this part of the larger puzzle. We're not trying to do signing that work well with the folks who are doing signing.

We're not trying to do an S-bomb in terms of provenance or where you got something from, when it was built, the system it was built on, that information or the security around that, but rather ensure that what we're building, the artifact tree itself, can be referenced and connected to from an S-bomb. It can be assigned and it can be referenced from a signed artifact. So you're not doing the signing itself. You're just facilitating saying, if you trust the CTO advisor signing authority, if you trust the CTO advisor signing authority, here's a link to the signature.

We're not even going to get in that space of delegating trust. Rather, if you're consuming a binary that's been signed by the CTO advisor, because you trust the CTO advisor, and that binary contains a fingerprint to a GitBomb tree, there is some extra metadata. So there's some extra data in that GitBomb tree that can give you information about that signed binary that you wouldn't otherwise have. And that information will help you if you want to reproduce it. That'll help you identify potential vulnerabilities in there, even in things that are nested in many layers of dependency deep across different languages that a normal S-bomb might not contain, right?

I love the goal of the project and I love the thoughtfulness into not getting into signing, becoming the central authority because there's so much already out there and it conflates the problem. And I'm still having, I'll have to actually go back and listen to the podcast again because I'm having trouble wrapping my brain around it completely because I'm used to this being from a proprietary software perspective, a couple of single entities. Like there's a signing authority and there's a certificate and then there's the repository from the proprietary software provider.

I am used to taking like a set of proprietary software and needing to then have a central repository. So, you know, whether it's Intel Landesk or some CA discovery tool, some authorities now maintaining a third-party database and saying, here's all my software and you're crossing your fingers, that here's all the software. And I trust CA or HPE or whomever to their limits of their ability, but, you know, how was that audited? And so the last question for either one of you, like, how is this audited?

How do I know that the repository, that this collective repository is accurate? So not so much accurate, because accuracy is a element, but it's not necessarily the thing that, if somebody signs something and the signature is wrong, you're not validating that. You're just saying, here's the data, but here's the metadata. The question is, how do I validate the accuracy of your metadata? It's a trust but verify issue. So we talk about GitBomb being a very non-greedy project. So for example, there are already projects like in Toto that are trying to provide a means for every different party that does something around the production of a stage of the software to sign it and attest that he did a thing.

So if I'm the person who built it, I can sign it and attest that I built the thing. If I'm the person who tested it, I can sign it and attest that I've tested the thing. And Toto does that very well. When you bring that together with reproducibility of builds, then you start to be able to ask questions like, if both Cisco and Microsoft have both built an open source project or a portion of the tree, and they've both signed it, do they agree?

Because if Cisco and Microsoft both agree, that is a lower probability, that's a higher probability of fidelity than if only one of them has made the attestation to you. If Google agrees with Microsoft and Cisco, their fidelity probability increases as well. Yeah. And then Ava, you were gonna give us a closing example of a log4j. Yeah. We've been asking about signing and trust, and that is a really critical part of this. One of the examples I love to give, what we want to create from this.

Just imagine, when the next log4j happens, most enterprises have some amount of software inventory management and hardware inventory management. And we need better SBOMs to be fed into that. But today's SBOMs only give you either a single layer of depth, who gave you this package or this proprietary piece of software, what version, what date, all of that stuff is good. Some SBOMs might have a little bit more depth, maybe two or three layers. If it's an open source project and it included a couple of libraries.

But I don't think any of them, none that I've seen go all the way back to source code files. With log4j, everybody struggled to find where in their enterprise, where across their fleet was this little tiny library embedded in a JVM, running in a web server, embedded in a JVM, running inside a piece of firmware, and had to rely on vendors to tell us that. Now, what if every piece of software you got, both open source and proprietary, came with a hash tree?

The hash tree is opaque, it's just hashes. It's not file names, there's no content. Vendors don't have to disclose what's in those files. But here's the hash tree. Then when there's an emergency and big security issue like log4j, CVE is released. MITRE or somebody else, national entity, then publishes, here's a list of file hashes that we know are connected to the CVE. And then every enterprise could cross-track their internal inventory management system, and it's just a join. List of hashes attached to a CVE, inventory of my fleet, and the hashes of everything in my inventory.

All of a sudden, you know that CVE is connected to a file somewhere in the tree of that piece of software you're running. And you still, you know, no one disclosed the source code and not saying that enterprise vendors should be sharing their proprietary stuff. But it would let your blue team or your incident response team find it almost immediately. I love that example. I can't tell you the number of times where we've used hacks in our inventory software to find this stuff.

2 is installed in a system. 2 isn't installed. 2, but that's how we found this dependency that we wanted to remediate. 2, and the intent was to find Adobe Premiere, whatever, whatever, with this vulnerability or et cetera. This fixes that problem because I'm looking for the, let's say that there's eight different versions or releases of log4j that are vulnerable. And I'm looking for these, this set of hashes across my entire portfolio. And now we just gotta do, you know, make sure that all the hashes are universal, blah, blah, blah, blah.

But that's a whole nother set of problems. But I love, love, love that example. If people want to find out more about GitBomb and how to contribute and get part of the conversation, because open source communities are never big enough, how do they do that? dev. That's our website. And from there, you can jump off to all the various community resources for plugging in. We have mailing lists, Slack channels, et cetera. You can also look for us at GitBomb on Twitter.

We're actually pretty good about posting things up there. We post our upcoming meetings. We post up the recordings of our community calls. So those are two really good onboarding points if you're interested in getting more involved in GitBomb. Avery, do you have anything you want to plug? The Twitter handles, projects, or is life all GitBomb right now? I've got so many irons in the fire, as it were, but GitBomb is one of the coolest things I'm working on. And if you want to catch either of us or get more info, I can't speak for Ed, but I've got a bunch of conference talks coming up where I'll be talking about GitBomb at Supply Chain Security Summit, Continuous Delivery Conference, and a bunch of other places in the next six months or so.

Ava, Ed, thanks again for joining the podcast. com. If you have a question that you can't somehow get to Ed or Ava to answer, I don't know how that's possible with all the information that you just gave. DM me on Twitter, at CTO Advisor is the Twitter handle. Talk to you next CTO Advisor podcast.