As application security and DevSecOps teams try to get the most bang for their app sec buck, one of the perennial problems has been figuring out where to focus their secure coding and vulnerability remediation efforts.
The scale of vulnerabilities that must be chased down in each application and the extent of the code that stretches across a typical enterprise portfolio of applications (and the enterprise's attendant software supply chain) make deciding on even the first step of prioritization a complex affair.
For the better part of a year, app sec veteran Mark Curphey and his business partner John Viega, who have depth of experience in the app sec space, heard variations on this complaint time and again. Curphey founded OWASP and had a hand in founding the companies OpenRaven and SourceClear, acquired by Veracode. Viega founded Secure Software, which developed the app sec testing tool Fortify, bought by Hewlett-Packard.
Before launching their next venture, Curphey and Viega went on a listening tour with top CISOs to get at their biggest app sec pain points. Here is what they learned in those conversations.
[ Join Nov. 14 Webinar: Secure by Design: Why Trust Matters | Learn more: Software Supply Chain Risk Report ]
Scan-and-fix is noisy and time-consuming
Curphey and Viega found general agreement that app sec leaders and their CISO bosses are faced with more and more noise as the complexity of scan-and-fix keeps rising.
"We sat down with about 100 CSOs and we basically said, 'We're going to do an app sec company, so that's the scope, but within that what's your biggest unsolved problem?' And what we heard very consistently was, 'We've got all the money in the world. We've got all the tools in the world. We can hire people. But we have massive amounts of noise, so we don't know what we should work on.'"
One recent study bolsters this point. In the survey, conducted among 300 security professionals by Backslash Security, 85% of respondents see being able to differentiate between real app sec risks and noise as critical to their success, but only 38% of them think their organization is able to do that. The biggest problem with their app sec tools? Almost half of the respondents (45%) said it was prioritizing findings, which takes a considerable amount of time given that they are "pretty noisy."
Curphey said that one CSO on their tour summed it up with a simple formulation:
"He said, 'If you could tell me what to work on now, next, or never, I'd write you a check right now.'"
The context gap
At many companies, security teams often run their tools on a piece of code without knowing whether that code is deployed in production, whether it has security fixes waiting to be deployed, and whether it is touching personally identifiable information (PII) or running critical business processes.
"All they have is this kind of myopic view of vulnerabilities."
—Mark Curphey
For many years, the only context available to organizations has been the criticality of vulnerabilities, but leaders seeking to prioritize their app sec efforts to gain better context about vulnerabilities — how they're deployed in the application stack, what kind of traffic is running through them, and how the application is used by the business. Curphey cited a game company he learned about during the tour.
"They have 6,300 repos, and they have no clue how many are in production. So they're scanning, and they're asking the developers to update the libraries in all 6,300. It's just noise. They don't need to be doing it. They don't know which ones are deployed into production, which ones have the highest traffic, or which one is touching PII."
—Mark Curphey
How to kill noise
Of all the contexts, knowing what is in production is one of the best ways to kill noise, the security leaders told Curphey.
"I think everyone historically — me included with SourceClear — focused on vulnerable libraries, and absolutely they are a problem. But the reality is that most of the vulnerable libraries never get put into production, so therefore it is just noise. You don't actually have to deal with it."
—Mark Curphey
The point was brought home by the CISO of a large financial services company.
"They didn't know what repos were in production but they knew the vast majority weren't, so they told me, 'The first thing we'd like to do before we do anything is just kill the noise at the source so that we don't have to figure out deduping and all of that other stuff.' Their take was, 'I can get rid of 95% of the problems right off the bat with that knowledge.'"
—Mark Curphey
Most SBOM efforts are still busywork
The general consensus from the listening tour on White House Executive Order 14028 on software bills of material (SBOMs) was that it is creating busywork for CISOs.
"You can generate one with your tool, but it's basically like writing your own doctor's certificate and signing it off yourself. It's totally easy to fake. Then, if you have one that happens at the build system, it is usually completely different than the one that's in the code repo and different from the one that runs in production, where you have hot loading and things like that."
—Mark Curphey
SBOMs are useful when you have attestation "and you have confidence that this SBOM is generated here, that one there, and so on," Curphey said. He explained that the fire drill of Log4j showed weakness in the narrative about SBOM usefulness.
"Most people sent folks around with clipboards trying to find out where it was. And even when it was in production, they found it was in test directories where it wasn't being called into production. They spent all of this time, shut down dev teams, and I think there's scar tissue for a lot of CISOs, who are saying, 'We can't rely on a lot of this. We can use [SBOMs] to help feed things, but ultimately they're not a source of truth."
From tour to open-source tool
Many of the observations from Curphey and Viega's tour have been folded into what Crash Override is doing with the release of its open-source security tool, Chalk, which collects metadata from build artifacts to provide queryable graphs that can help app sec teams get visibility into what part of a codebase is in production, as well as the context of how things are deployed.
