A new study adds force to the argument that organizations need to look beyond vulnerability remediation when it comes to managing and mitigating software cyber-risk.
The study, by a Purdue University researcher, shows that the new Exploit Prediction Scoring System (EPSS), which many organizations are now using to prioritize vulnerability remediation, is not as effective as previously assumed.
The study demonstrates that, like other vulnerability risk assessment frameworks such as the Common Vulnerability Scoring System (CVSS) severity scores and the U.S. Cybersecurity Infrastructure and Security Agency's catalog of Known Exploited Vulnerabilities (KEVs), the EPSS is useful, but it is not a completely predictive mechanism for protecting against vulnerability-related threats.
Here's what you need to know about the limitations of the EPSS — and why your application security (AppSec) strategy needs to extend to more modern approaches.
[ Download Today: 2025 Software Supply Chain Security Report | Join the SSCS Report Webinar ]
Predictive models don't tell the entire story
The EPSS, which is maintained by the Forum of Incident Response and Security Teams (FIRST), is a vulnerability scoring system that uses machine learning (ML) to predict the likelihood that a software vulnerability will be exploited in the wild. The goal of the EPSS is to help security teams prioritize patching efforts based on actual exploitation risk rather than just severity scores. Many organizations see the EPSS as significantly lowering software risk.
Ken Dunham, cyberthreat director at Qualys' threat research unit, said that predictive models such as the EPSS are helpful indicators but never tell the entire story — and therefore should not be relied upon for the full context of how to identify cyber-risk. Risk is best identified using a variety of indicators and measurements, he said. That means that, in addition to using CVEs, CVSS scores, the EPSS, and KEVs, organizations making risk-mitigation decisions need to consider things such as attack surface, the criticality of assets, and environmental factors.
"The bottom line is that all organizations should have a risk-based process in place, personalized to their assets and architecture, to best identify and prioritize risk as the threat-scape changes and emerges. It's less about how to improve a predictive model and more about the process of how you manage risk using these predictive models as tools, and as only part of the equation."
—Ken Dunham
A trailing indicator of risk
In the Purdue study, the researcher, Rianna Parla, focused on high-severity bugs in CISA's KEVs, comparing each against its EPSS score history to see whether an EPSS score was a good indicator of whether a vulnerability would be exploited within the next 30 days. The analysis showed that in some cases the EPSS predicted a CVE's probability of exploitation before the vulnerability landed in the CISA KEV catalog, but in many other cases, it did not.
Parla's conclusion: The EPSS is more of a trailing indicator than a predictive system:
"The Exploit Prediction Scoring System name may be a bit misleading as it is less of a prediction system and more of a risk scoring system. In other words, what is the risk of exploitation of a given vulnerability versus other vulnerabilities. Most of the analyzed data showed that EPSS was less effective as a measurement than simply using the CISA KEV catalog directly."
Parla's results should be of note to the many organizations that, knowing that it is impossible to remediate every single vulnerability in their environment, use the EPSS to prioritize the risks they need to address first. Unlike CVSS scores, which measure the potential severity of a vulnerability's impact, the EPSS is supposed to predict the likelihood of attackers actively exploiting the vulnerability, based on a combination of real-world exploit data, threat intelligence, and ML.
Context is key
Jeff Williams, co-founder and CTO of Contrast Security, said the key to unlocking any benefits from such mechanisms is context. That context, he said, consists of technical details on what systems are being targeted, what attack vectors are being used, and which kinds of vulnerability are being discovered by attackers, as well as business factors such as which systems are attractive to attackers, what kinds of data is present, and what access an exploit could provide.
"If you only look at a score that hasn’t been enriched with context, then you’re never going to get good predictions."
—Jeff Williams
Parla wrote that the results of the study highlight the need for a defense-in-depth approach to cyber-risk mitigation. Though occasionally the EPSS does correctly identify high-risk vulnerabilities prior to their inclusion in the CISA KEVs, organizations cannot solely rely on it as a risk-mitigation approach, she said.
Though the study shows that the EPSS is less effective than assumed, no one is suggesting that organizations stop using it or any of the other vulnerability risk-assessment mechanisms. Security experts perceive each framework — CVSS scores, the EPSS, and KEVs — as providing at least some help in guiding organizations to vulnerabilities that present the highest risk and therefore take priority. But the key is to keep expectations in check and understand that a focus on vulnerability patching alone is risky.
John Bambenek, president of Bambenek Consulting, said that while organizations need all three risk-assessment frameworks to inform software risk-mitigation decisions, the best intelligence will always be whatever attacks and malicious activity the organization sees on its own networks and what attackers have targeted. The CVSS framework was not designed to tell you what an attacker will do, Bambenek noted, and KEVs only have data on attacks and vulnerabilities that are relevant to CISA's core audience of federal government agencies. He called the EPSS a good attempt to get to something that is more predictive, but he added that trying to predict the future is always difficult.
"Using AI/ML for cybersecurity is an inherently risky thing because attackers already have decades of experience in fooling automated systems and their entire mode of operations is doing unpredictable things. Using retrospective models for future prediction breaks down when you are trying to model behavior of those who don’t have to worry about technical debt or reverse compatibility."
—John Bambenek
Inherent limitations with ML
Casey Ellis, founder at the crowdsourced cybersecurity firm Bugcrowd, said ML-driven predictions have inherent limitations that make them effective at ranking known threats but not so much at predicting unknown exploits. "To refine these models, integrate broader datasets, consider advanced analytics such as large language models for deeper context, and regularly validate outcomes against real-world incidents," Ellis said.
SOCs and incident response teams should prioritize the KEVs and flaws listed on CISA's catalog as active threats, Ellis said. They should also prioritize vulnerabilities based on CVSS scores and use EPSS scores to identify vulnerabilities with high likelihood of exploitation, even if not yet listed by CISA, he said.
Beyond that, look at asset criticality and maintain your own threat model, Ellis said.
"Prioritize based on the importance and exposure of affected assets. Regularly reassess and adjust your prioritization strategy as new threats and data emerge."
—Casey Ellis
Why your AppSec must shift beyond vulnerability mitigation
Nearly a quarter of those CVEs were exploited on or before their public disclosure, meaning organizations had little or no opportunity to patch the vulnerabilities before attackers began exploiting them. Making matters worse, last year saw a 40% increase in total vulnerability disclosures, reaching 39,000, up from 28,000 in 2023, a new report from GuidePoint Security disclosed. That's an average of 378 vulnerability disclosures every day.
These reports — and the rise of artificial intelligence-derived coding and software supply chain attacks — demonstrate that the time has come for organizations to look beyond vulnerability mitigation when it comes to shoring up their AppSec — and take steps to develop a modern approach.
