The first analysis of software supply chain security based on the Open Software Supply Chain Attack Reference (OSC&R) threat framework has been released, and the news isn't good. "While [application security (AppSec)] programs and practices continued to mature in 2023, our analysis indicates there is much more work needed if we are to manage the risks effectively," the report, "OSC&R in the Wild: A New Look at the Most Common Software Supply Chain Exposures," said.
"Detecting and remediating legitimate security risks from within the mountain of benign code-hygiene alerts issued by traditional AppSec technologies continues to be problematic, and serious vulnerabilities are passing to production code with concerning regularity."
—OSC&R report
The OSC&R report's revelations are important for both AppSec risk managers and development teams to understand. Here's what your teams need to know.
[ Get RL's Essential Guide: Software Supply Chain Security for Dummies ]
'OSC&R in the Wild': Key takeaways
Here are the high-level points to draw from the report:
- AppSec teams face an unmanageable volume of alerts, with the average AppSec team monitoring 129 applications and triaging over 119,000 security alerts annually.
- Most organizations face high-severity risks, with 95% of organizations having at least one high, critical, or apocalyptic risk — the three highest severity rankings — within their software supply chain, and the average organization having nine such issues.
- One in five applications contain runtime exposure, with 20% of all applications having high, critical, or apocalyptic issues during the execution stage of OSC&R.
- Older vulnerabilities, such as command injection (15.4%), sensitive data in log files (12.4%), and cross-site scripting (11.4%), are still the most common in applications.
- Six of the top 10 most commonly observed vulnerabilities are tied to poor implementation of fundamental security practices, such as authentication, encryption, exploitable information in logs, and the principle of least privilege.
- Automated, contextual analysis can dramatically reduce the volume of overall alerts by more than 97%, accelerating the identification of the critical alerts organizations need to address.
AppSec alert fatigue is a real problem
One of the questions researchers sought to answer was whether there was alignment between the vulnerabilities found in the wild and the focus of AppSec teams, OX's CEO, Neatsun Ziv, said.
“The data suggests there is a misalignment. We found significant vulnerabilities at every stage of the kill chain.”
—Neatsun Ziv
The cyber kill chain outlines the stages of a common cyberattack, from early reconnaissance to data exfiltration. OSC&R, modeled along the lines of the MITRE ATT&CK framework, describes the tactics, techniques, and procedures (TTPs) used by attackers on software supply chains. It was introduced in 2023 and developed by OX, Microsoft, Oracle, GitLab, Fortinet, FICO, and others.
Over a nine-month period, OX’s researchers analyzed nearly 140,000 enterprise applications and correlated the data to OSC&R. “With OSC&R’s kill chain and TTPs specific to software supply chains, we show how an adversary views a software supply chain environment: as a prize to be taken, likely ripe with vulnerabilities if they look hard enough,” the researchers wrote in the report.
Liad Cohen, one of the authors of the report, said supply chains are complex ecosystems with innumerable interconnected components — and that has led to AppSec alert fatigue.
“This complexity generates a massive amount of alerts, and on top of it, a massive number of false positives when, or if, the right tooling isn't used to detect flaws and weaknesses. Due to the sheer volume, accompanied by time pressures and resource constraints, developers can't possibly address every alert, especially if prioritization isn't attached."
—Liad Cohen
Roger Grimes, a defense evangelist with KnowBe4, said bloated alert volumes are a huge problem for security teams.
“All software and services should spend far more time and resources on determining what is or isn't a critical problem that needs to be researched or addressed,”
—Roger Grimes
Chris Morales, CISO of Netenrich, said alert fatigue is a "massive headache for supply chain defenders, and it's easy to see why."
"Imagine trying to spot a needle in a haystack, except the haystack keeps growing by the minute. That's what our AppSec teams are dealing with. Monitoring an average of 129 applications and sifting through over 119,000 alerts every year is overwhelming, to say the least.”
—Chris Morales
Software complexity explains the high-severity risk problem
The large amount of high-severity risk the researchers found in software supply chains can be attributed to factors such as the complexity of today's software development. “Large and complex codebases introduce vulnerabilities into the supply chain,” said Ronen Atias, also an author of the OSC&R report.
Atias said that legacy scanning and vulnerability assessment tools might not be able to dive deep enough into codebases to identify some compromises. Another contributor: The widespread use of third-party components.
“Developers often rely on third-party components which may include unknown or undetected vulnerabilities that impact applications.”
—Ronen Atias
Risk can also be introduced by security tools themselves, Atias explained.
“Many organizations use disparate and siloed tools. Then they have to piece together the results, which is time-consuming and error-prone. An integrated approach that also includes automated remediation is really what's necessary."
—Ronen Atias
Robert Schiela, technical manager of cybersecurity foundations at Carnegie Mellon University’s Software Engineering Institute, said that even if an organization knows its software has risks, it may not be able to remove them. “Just because an organization knows there are problems doesn’t mean there are mitigations that it can perform or afford,” Schiela said.
“They may be waiting for software vendors to provide security updates. Even if the software is an open-source program that relies on a dependency that has a security vulnerability, it would be costly and may not be reasonable for an organization to maintain its own copy of the program and update the code itself.”
—Robert Schiela
Ken Dunham, cyberthreat director in the threat research unit at Qualys, said that in some cases, the complexity of legacy and hybrid infrastructure with complex dependencies and contractual requirements may force difficult risk decisions, resulting in accepted risk for extended periods of time.
"Over time, accepted risk becomes unacceptable and must be remediated.”
—Ken Dunham
The game-changing nature of runtime errors
Runtime errors, which open an application to attack during OSC&R’s execution phase, are also posing a significant threat to software supply chain security, the report said. "Runtime issues are critical to an app's security because they represent vulnerabilities that can be exploited while the application is actively running,” OX’s Cohen explained.
“Unlike static code analysis, which examines code before it's executed, runtime issues emerge when the code interacts with real-world data and conditions. These issues can lead to unpredictable behavior, data manipulation, race conditions, and zero-day exploits, all of which significantly increase risk, if not handled swiftly and properly."
—Liad Cohen
Qualys' Dunham said threats in the world of runtime move into the world of fileless and in-memory, which can be a game changer for traditional security tools to detect and prevent.
“This often forces organizations to consider behavioral and zero-trust solutions to better remediate such threats, where traditional tools may fail to detect and respond.”
—Ken Dunham
MJ Kaufmann, an author and instructor at O'Reilly Media, said inherited risk is a serious concern.
"No matter how well an organization writes its own code and assesses it for security issues, the application inherits risk from the libraries it imports, which creates runtime issues, leaving the application vulnerable despite the best efforts by developers."
—MJ Kaufmann
One of the more discouraging findings in the report is the persistence of old vulnerabilities plaguing the supply chain, including command injection, storing sensitive information in log files, and cross-site scripting, said Julian Brownlow Davies, vice president for advanced services at Bugcrowd. One key culprit: The pressure to quickly deliver new features can lead to security being overlooked, and the complexity of patching these issues can cause organizations to defer updates.
“These vulnerabilities persist for a few reasons. Developers may not fully understand their implications or how to mitigate them. Older codebases, which aren't regularly audited, can continue to harbor these issues. There's also often a lack of proper training in secure coding practices."
—Julian Brownlow Davies
KnowBe4’s Grimes said that patching and training issues are contributing to the persistence of common vulnerabilities. He explained that unpatched software and firmware is the second-most popular initial root access for hackers' malware creations breach data. “Clearly, organizations are not doing enough to quickly identify and patch vulnerable systems,” Grimes said.
As for training, he continued, a big part of the problem is that almost no programming school teaches Secure by Design programming. “How can we expect programmers to be great at secure design and at not putting in critical flaws and vulnerabilities when we literally don't teach them?” he asked.
“The reason schools don't teach it is that employers don't request it. If more programmer's job descriptions and want ads included a requirement that the programmer be knowledgeable in secure design and programming principles, we'd have far less bugs and vulnerabilities. But employers don't, and I have no idea why not.”
—Roger Grimes
Netenrich’s Morales acknowledged that more rigorous training is needed, but he added that security needs to be baked into the DNA of organizations. “We need to create a culture where everyone, not just the security team, understands their role in maintaining a secure environment,” Morales said.
Organizations struggle with software supply chain security
Equally discouraging is the researchers’ finding that more than half of the top 10 most commonly observed vulnerabilities are tied to poor implementation of fundamental security practices. “Many organizations are doing the best they can, but they are hampered by legacy AppSec tooling that wasn't built for today's speed of development and overwhelming volume of issues,” said OX’s Cohen.
“Looking at these findings, it's clear that many organizations are struggling with Security 101. When we see vulnerabilities tied to basic security practices — things like weak authentication or poor encryption — it's a red flag that fundamental security hygiene isn't up to scratch.”
—Liad Cohen
Cohen said that in his view as a CISO, this problem calls for a back-to-basics approach in security operations. “It's not always about the latest shiny tool. Sometimes, it's about doing the fundamentals right, consistently,” he said.
One of the promising findings in the report is the effectiveness of automation in addressing alert overloads. According to the researchers, alert volumes can be greatly reduced by applying automated consolidation, deduplication, and contextual analysis. In applying those processes, the researchers reduced the average volume per organization from 118,000 raw alerts down to 666 high-priority issues. “This is still a significant number, but much more manageable,” the researchers wrote.
Carnegie Mellon University’s Schiela said AppSec alert automation can be effective when configured correctly, but he warned that it adds a new layer of complexity for development and security teams. “It’s another tool to help more precisely configure the false-positive and false-negative tradeoff but can increase risk if not tuned correctly for your risk profile,” Schiela said.
Fewer alerts and more actionable insights are the answer
Automation, along with artificial intelligence, are at the heart of an industry transformation toward efficacy and efficiency, said Qualys' Dunham. “Proper design and orchestration to SIEM and SOAR outcomes make it possible to identify false positives, with events that can be auto-processed and those that can be automatically prioritized to the top of the queue for management by an analyst,” he said.
Netenrich’s Morales said automated alerts, especially when coupled with contextual analysis, "are game changers in cutting through alert noise,” added Morales. But he noted that there is a kicker.
“It's not just about reducing noise. These automated systems provide actionable insights crucial for making quick, informed decisions. In cybersecurity, where every second counts, having this level of clarity can make all the difference.”
—Chris Morales
Caroline Wong, chief strategy officer for Cobalt Labs, cautioned about the limits of automation. She said some of the most critical risks require manual effort by a technically skilled security professional to identify. Business logic flaws and chained exploits are two examples, Wong said.
“If organizations are not proactively performing manual security testing on their assets and relying only on automated defect discovery, these critical vulnerabilities might fly under the radar and go without getting addressed or remediated for long periods of time.”
—Caroline Wong
The researchers noted that their report makes clear that there’s still a long way to go before the problem of vulnerable software supply chains is solved:
“Progress is being made, but the high volume of vulnerabilities that are passed through the supply chain into live applications and the large percentage of organizations reporting supply chain security incidents, both indicate that greater focus could be put on prevention rather than on detection.”
