ReversingLabs Chief Software Architect Tomislav Pericin discusses the recent NIST guidance on secure software development and software bill of materials.
PAUL: Hey, I'm Paul Roberts and I'm the Cyber Content Lead here at ReversingLabs. If you're part of a software development organization that does business with the federal government, there's some pretty important news that dropped last week that you should know about. NIST, the National Institute of Standards and Technology released some guidelines on what it calls Secure Software Development Framework. These are basically guidelines that companies which license software to the federal government are going to be expected to follow to make sure that their software is secure.
This pretty complex stuff, so I sat down with Tomislav Pericin, who's the Chief Software Architect here at ReversingLabs and an expert on software supply chain security, to talk about what's in the new guidance, and how companies should be responding to it. If you like what you hear us talking about, you might want to check out the new white paper that ReversingLabs has released. It explains the new guidance from NIST on Secure Software Development Frameworks and some of the recommendations around things like Software Bill of Materials, you can find a link to it in our presentation. Thanks for listening.
PAUL: Tomislav, welcome, and thanks for taking time out of your day to talk about this. You are the Chief Software Architect and co-founder at ReversingLabs.
TOMISLAV: Hi, Paul, nice to be here.
PAUL: It's great to have you.
PAUL: So we're talking today about some news that came out really late last week, relating to a topic that is near and dear to your heart and ReversingLabs’ heart, which is secure software development and software supply chain. And the news was some publications by NIST, the National Institute of Standards and Technology, in response to an Executive Order that President Biden put out back in May, calling for greater software supply chain security. So I thought I'd actually start by asking you kind of the big question, which is “Why?” “Why are we getting this kind of guidance, suddenly from the US government on something as down in the weeds as software development and software supply chain?”
TOMISLAV: Well, it must feel like it's all of a sudden, but, legislation and executive orders like this, they take a while to be prepared. And some semblance of what's been prescribed by the Executive Order has been in the works for the last almost a decade, really. So one of the key things in those, in that Executive Order was the mentioning of something called Software Bill of Materials, which is a list of ingredients really, for any software package. And the reason why that matters, is because the end user of that software product needs to understand what does the software consist of, so they can better model the risks. So again, track vulnerabilities within those products. And more importantly, as of late, to understand if there's a software supply chain attack, are they exposed to this particular attack? You know, what kind of impact can the attack have on their environment, and things like that. So it really centers around a very good set of understanding of what the software is composed of, to actually build all this knowledge base, what's installed, when it's installed and what the underlying impact of any vulnerability could be.
PAUL: Right, and these executive orders, so they're not creating policy, but what they basically do is, is give orders to executive agencies - NIST is part of the Department of Commerce, for example - to say ‘within this amount of time, worry, we want to see this from you.’ Right. And so this announcement last week is really in response to a deadline that was set out back in May in saying, ‘we want to see some specific guidelines around secure software development, software supply chain.’ I guess the idea is to make the software that the federal government uses more secure. Could you kind of just connect the dots? How does focusing on things like Software Bill of Materials or secure development, improve the security posture of federal government networks and agencies?
TOMISLAV: Yeah, so NIST came up with a document about secure software development practices, it's called a framework, because it kind of outlines what are the necessary steps to actually build secure software releases. There are a couple different categories that are laid out in the framework. You know, ‘how to deal with the vulnerabilities on the source code level;’ ‘how to identify software components through using either software composition analysis tools for both source code, and binaries.’ There's the best practices to have the build environments, and how to track things through (a) Software Bill of Materials and how to secure the software supply chain. There's a really large body of work, trying to be condensed into set of really prescriptive guidances to, first and foremost, software developers because ultimately, any kind of problem, which is discovered within the software supply chain, can first be addressed by the software developer and the publisher of that software application. And then for consumers of this software, it is also guidance on how to isolate and how to manage software releases; how to track (and) create inventories and things like that. It's a really comprehensive set of guidelines. And it will be great if they are studied across organizations, and some parts of this, at least, start to be implemented sooner rather than later.
PAUL: So, I mean, the context for this, obviously, is in the last year, we've had a number of incidents that relate to software supply chain security, the first and foremost SolarWinds, the hack of SolarWinds, reportedly by, nation-state actors, maybe affiliated with Russia. But there have been other incidents as well, there was a CodeCov. And then of course, going back, much, much further CCleaner (and) NotPetya. So these types of attacks have been a growing trend, I guess. How does this guidance connect back to those incidents and to those problems - the problems that were identified? For example, with SolarWinds?
TOMISLAV: Yeah, at the very base level, Software Bill of Materials allows you to answer a question: ‘What do you have installed?’ And if you have an inventory, or a full Software Bill of Materials for all of the products that you're actually using in your environment, you will be able to answer that question relatively quickly. And in the early days of announcing the SolarWinds hack, and very recently, the Log4J vulnerability as well, they are both pointing to the Software Bill of Materials as this key knowledge base place where you can actually ask a question, and immediately learn about the exposure. And that's why a lot of the language in the Executive Order, and all these in these documents, they always want to point out the Software Bill of Materials, because it is key to have it.
PAUL: So one of the big releases last week, like you said, was basically they took this framework document, this draft document and made it official. So there's now this document, SP Special Publication 800-218, that is this secure software development framework or SDF, that is now basically an official NIST government publication. That document does a number of things, but one of them is to talk about the obligation that software producers, so companies that make software that the government buys or licenses have to do, and they talk about obtaining provenance data for the components of their software releases, could you just sort of talk about what they mean by provenance data? And what types of things software producers are being asked to account for?
TOMISLAV: Yeah, modern applications are very complicated. They consist of thousands of different components. And they're kind of built together at a rapid pace, because that's kind of how the market demands applications to be built nowadays. And it's not uncommon to include components into your build environment and automated product, that they have not been fully vetted. Right? So if you think about the modern web application that's been built on open source package repositories like NPM, easily you get in a state where your web application consists of 10,000 different modules just being downloaded from a public repository. The software supply chain attacks have started to target these open source repositories as well. And that's exactly what the point of the question ‘Do you have provenance data?’ So do you understand who is behind these components? Do you understand that there are actually legitimate components that have not been tampered with? They haven't added any malicious code to them, or anything like that. Software Bill of Materials gets you part of the way there because at least you know what's included?
PAUL: Yeah.
TOMISLAV: But this bigger scrutiny of what are you actually using, not only from the compliance standpoint of ‘are you licensed’ compliant, right? ‘Are you using it in a way that was intended?’ But more in the sense of ‘Is this going to have a negative effect on any of the users?’ From a vulnerability standpoint, yes, but also from the software supply chain attack stand point. Now there's this huge surface area of risk, which is largely an audit that, and it needs to be accounted for by the publisher as if they were publishing that code themselves, because ultimately, the users are running that code.
PAUL: So even if you're making an ingredient, in essence, that goes into some bigger recipe of a software application, you still need to make sure that that is a high quality ingredient that it's not going to compromise the larger project. And I think you raised an important point, which is, I think, like we often in cybersecurity have focused on vulnerabilities, software vulnerabilities, because those can open doors to attacks. And that's definitely a focus of this – reducing the number of software vulnerabilities. But there's a larger issue too, which is actually just outright malicious code: seemingly legitimate components that have actually been compromised, either open source or proprietary. And this was an issue that came up in the SolarWinds incident. Can you just talk a little bit about that particular incident, and this difference between focusing on software vulnerabilities and actually trying to identify malicious components, or compromising components?
TOMISLAV: SolarWinds basically, as you know, was an attack where there was a backdoored piece of code added to a legitimate code base, SolarWinds Orion, which for the most part is a closed source application. So the ability to audit it is limited to those who can analyze the binary components without any access to source code. But more importantly, it's not a vulnerability itself. It's a backdoor.
What this NIST document called Secure Software Development Framework (SSDF) has laid out is the responsibility to actually audit the behaviors of those applications, and to ensure that there's not been any malicious tampering. In essence, it’s trying to kind of equate the responsibility of anybody using our closed source component to say, this is as if you're publishing that part of your software yourself, right? In this particular case, for SolarWinds, that would have meant, obviously, they have source code, and they've been running a whole bunch of analysis tools on there, but the build environment was tampered with. And the backdoor was added during build. And what the SSDF implies is that the release code is something that users are actually going to run, and that it needs to be checked for tampering as well. There's quite a few places in that document where they delineate between source code and binary. And you are literally saying that both are required, because the users are actually running binary code.
PAUL: Right, and that's an important thing to note is that the guidelines or the guidance from NIST actually says both ‘yes, do source code analysis,’ but also that you should be testing your binaries, your compiled code, before you ship them to make sure that you're not shipping malicious code. And that seems to be something that really speaks very specifically to what we saw with the SolarWinds compromise. Do most software publishers, software producers do that right now?
TOMISLAV: Well, we ran a survey relatively recently, which was in a limited capacity just to understand what's the current state of the art when it comes to protecting software releases. And it's not encouraging what the results showed, there's a limited amount of testing. It mostly is done by legacy technologies that are trying to solve a completely different problem. So if you're doing antivirus scans on software releases, yes, you can prevent some types of issues. But issues like the SolarWinds attack, and these maliciously added backdoors are not caught using either signature technologie (or) machine learning technologies, this is a completely different type of problem, which, at its core, kind of dictates that you do have to understand, the tampering of binaries from a different standpoint So literally you need to be able to audit, what's going to happen when you run this application and to see is there any change between this and the next version or in between builds, or things like that. So, a lot of people are gravitating to off-the-shelf solutions, and, it feels like a checkbox answer more than trying to solve for the problem that has been widely talked about with the SolarWinds hack.
PAUL: So one issue is attestation which is: who gets to decide that they are complying with the guidance that NIST has provided? And basically, the position that NIST took was, by-and-large, the government, which is the audience for this document, should let software producers self attest as to whether they're compliant, it’s called “first party attestation.” Unless they have a really compelling reason not to, either attest it themselves, second party or hire a third party to do the attestation for them. So I'm interested in your thoughts on how well you think it will work to have the software producers basically saying how compliant they are or aren’t with these types of guidelines.
TOMISLAV: Well I think it's a good first step, right? I don't think that's the final state of it, to be honest, because at this stage we have most of the companies who are producing software that are not producing Software Bill of Materials at all and those that are are mostly doing them by hand, so that they say they're compliant with the license agreements for the software components they’re using. So this wide adoption of Software Bill of Materials is still something that needs to happen, and recent surveys do show that a lot of the companies – an overwhelming majority – are interested in both producing and consuming them, which is a great step.
So if we have that then initially having people self check or validate is okay, but in the long run it does lead to third party oversight because there's a bunch of different ways how you can actually create a Software Bill of Materials and even though different government documents like the NTIA document, which kind of outlines what the minimum requirements for the Software Bill of Materials, actually exist. They do recognize the difficulty of producing a well produced Software Bill of Materials, and they cannot account for the gaps in it, right? So if that is something we expect, that there's going to be gaps and the different technologies will produce different results, then at one point self attestation stops making sense, and we need to have kind of more defined set of guidelines of what constitutes a really good Software Bill of Materials, how it's validated, and who can actually do that reliably. Because you can have different results if different people are doing the measurement, and that's kind of the problem when you're trying to trace all of these components and you assess them and the rest of it. But it's a good first step, but I think more needs to be done.
PAUL: Okay so the second question or concern is that I think a lot of people were expecting that at the same time as they release this guidance on secure development framework, that they would also release some final guidance around the Software Bill of Materials themselves. There's a separate NIST document, this 800-161 document that they’ve included some draft language around, what they want to see with Software Bill of Materials, what they should and shouldn't do and so on, but that remains in draft form, that’s still not finalized language. For companies that want to obviously are software producers selling it to the federal government, know now that they’ve got their orders to produce the Software Bill of Materials, but the language is still kind of, not finalized. Where does that leave them and what are your recommendations on what they should do?
TOMISLAV: Well, I’d start learning how to produce a Software Bill of Materials consistently for every software release, depending on your release cadence that could even be coming directly out of the CI system as you're building the software product. That's kind of the ideal state, because for every package you build you’ll have the Software Bill of Materials handy that you can give out to anybody who needs it. Or you can include it in the release package. So it's really handy, but getting through that state is going to take a while.
If you start reading the document, they do kind of anticipate many points when this can be done. I think the easiest one to start with is from the binary analysis standpoint, because there are less integration points that you need to worry about. They do kind of anticipate you needing to do that on the source code level as well because you can do code snippets that you can copy from other projects, have header-only libraries and the rest of it. It's a massive problem, which is why I feel it’s still in the draft mode, and it’s probably going to be released in stages, just because of that. I think the idea right now is to get as many people on board as possible to start to get the value out of these systems and then start refining as you kind of see it being used in practice.
PAUL: Okay now final question or concern is with the scope of this new guidance and NIST said in sort of a fact they just reiterated these guidelines apply to software producers selling software to the federal government, they are not meant to apply to software government agencies doing development themselves internally. So none of these guidelines and guidance applies to the software development shops within agencies that are producing applications and so on. Thoughts on that? Sounds to me a little bit like “do as I say but not as I do,” but what are your thoughts?
TOMISLAV: I think that Secure Software Development Framework is a nice piece of work, and it can be applied regardless of who’s doing software development, right? I think that's kind of the maturity stage of the organization, but that's something you want to strive for. It should be applied uniformly across the board. I see reasons for exceptions like this software's being developed with best practices in mind, regardless of who’s developing it. And it's probably just the lack of time really from the executive board to being able to commit to actually doing it. But I foresee that in the long term it will be applied on both ends. Like even for producers of the software, even if the software's been developed by the government itself. I don't necessarily foresee that a lot of those things will be public, right? Software Bill of Materials for internally used software does not need to be released. But as the government starts ingesting large numbers of Software Bill of Materials, and having a nice place to actually ask questions of importance, then obviously, having inventory of your own written software is also handy, so I think it's just a matter of time.
PAUL: A lot of this comes down to the maturity of the organization and how hard it's going to be for them to implement a Software Bill of Materials. What are your thoughts on the best way for organizations to get it into a position where they can implement this?
TOMISLAV: Start now, really, I mean there's all these frameworks in place, there’s a bunch of documents, software composition analysis - as a set of tools - has existed for quite a while. You know, start using things that are already available and I think you’re going to be at a much better place. You're gonna start understanding what your software consists of, and there’s a bunch of learning that’s just from the vulnerability analysis and remediation standpoint there which could be used to improve the products already. So start with that and then start expanding to areas of software supply chain protection, understanding the provenance of these packages and enforcing integrity checks across the board really, so you can ensure that the delivery mechanism also is covered by this. So a lot needs to be done starting now before it's mandated, I think it's gonna be way better.
PAUL: And I know ReversingLabs, we've got a white paper and some other supporting materials out, we'll be linking to some of these documents that were released and guidelines that were released last week. But any other suggestions on where organizations can look for help as they're trying to stand up this Secure Software Development and SBOM infrastructure?
TOMISLAV: Yeah I mean the team at ReversingLabs has really dug into all of these documents and this white paper that’s out now - as you’re viewing this - is a good reference point. We’ve summarized everything that NIST has done, summarized the Executive Order, Secure Software Development Framework has been touched upon, I think NTIA document, the minimum requirements for SBOM as well. So if you're feeling lost and there's a lot of information, I think that's a good starting point.
PAUL: Yep.
TOMISLAV: It’s a relatively short document if you will, start with that and that gives you all these handlings to all of these frameworks and ways to think about Software Bill of Materials and the software supply chain protection in general.
PAUL: And as you said this is a work in progress, so there’s going to be more announcements, and this is something that’s evolving, both on the federal government side obviously and in the commercial software development marketplace as we speak. But, Tomislav Periĉin, Chief Software Architect at ReversingLabs, thank you so much for coming in and speaking to us about this. I'm sure we’ll hear from you more on this in the future.
PAUL: Hey, thanks so much for listening to this conversation with myself and Tomislav Periĉin about NIST’s new Secure Software Development Framework Guidelines. Like I said there's plenty more information on the ReversingLabs website. We've got a White Paper explaining the new NIST guidelines around the Software Bill of Materials. We’ve also got a blog post that breaks down what NIST announced in response to the Cyber Executive Order in May, and what it means for your company. Check it out and some of the links are below.
