{"id":25620,"date":"2024-12-30T09:20:54","date_gmt":"2024-12-30T17:20:54","guid":{"rendered":"https:\/\/www.palada.net\/index.php\/2024\/12\/30\/news-19348\/"},"modified":"2024-12-30T09:20:54","modified_gmt":"2024-12-30T17:20:54","slug":"news-19348","status":"publish","type":"post","link":"https:\/\/www.palada.net\/index.php\/2024\/12\/30\/news-19348\/","title":{"rendered":"Prioritizing patching: A deep dive into frameworks and tools – Part 2: Alternative frameworks"},"content":{"rendered":"

Credit to Author: Matt Wixey| Date: Mon, 30 Dec 2024 15:05:30 +0000<\/strong><\/p>\n

\n

In the first part of this series<\/a>, we took a close look at CVSS and how it works, concluding that while CVSS may offer some benefits, it\u2019s not designed to be used as a sole means of prioritization. In this article, we\u2019ll cover some alternative tools and systems for remediation prioritization, how they can be used, and their pros and cons.<\/p>\n

Exploit Prediction Scoring System (EPSS)<\/h1>\n

EPSS, first published at Black Hat USA 2019<\/a>, is (like CVSS) maintained by a FIRST Special Interest Group<\/a> (SIG). As noted in the whitepaper<\/a> that accompanied the Black Hat talk, the creators of EPSS aim to fill a gap in the CVSS framework: predicting the probability of exploitation based on historic data.<\/p>\n

The original version of EPSS used logistic regression: a statistical technique to measure the probability of a binary outcome by considering the contribution several independent variables make to that outcome. For instance, if I wanted to use logistic regression to measure the probability of a yes\/no event occurring (say, whether a given person will purchase one of my products), I\u2019d look to collect a large sample of historical marketing data for previous customers and would-be customers. My independent variables would be things like age, gender, salary, disposable income, occupation, locale, whether a person already owned a rival product, and so on. The dependent variable would be whether the person bought the product or not.<\/p>\n

The logistic regression model would tell me which of those variables make a significant contribution to that outcome, either positive or negative. So, for example, I might find that age < 30 and salary > $50,000 are positively correlated to the outcome, but already owns similar product = true is, unsurprisingly, negatively correlated. By weighing up the contributions to these variables, we can feed new data into the model and get an idea of the probability of any given person wanting to buy the product. It’s also important to measure the predictive accuracy of logistic regression models (as they may result in false positives or false negatives), which can be achieved with Receiver Operating Characteristic<\/a> (ROC) curves.<\/p>\n

The creators of EPSS analyzed over 25,000 vulnerabilities (2016 \u2013 2018), and extracted 16 independent variables of interest including the affected vendor, whether exploit code existed in the wild (either in Exploit-DB or in exploit frameworks like Metasploit and Canvas), and the number of references in the published CVE entry. These were the independent variables; the dependent variable was whether the vulnerability had actually been exploited in the wild (based on data from Proofpoint, Fortinet, AlienVault, and GreyNoise).<\/p>\n

The authors found that the existence of weaponized exploits made the most significant positive contribution to the model, followed by Microsoft being the affected vendor (likely due to the number and popularity of products Microsoft develops and releases, and its history of being targeted by threat actors); the existence of proof-of-concept code; and Adobe being the affected vendor.<\/p>\n

Interestingly, the authors also noted some negative correlation, including Google and Apple being the affected vendors. They surmised that this may be due to Google products having many vulnerabilities, of which relatively few were exploited in the wild, and Apple being a closed platform that threat actors haven\u2019t historically targeted. The inherent characteristics of a vulnerability (i.e., the information reflected in a CVSS score) appeared to make little difference to the outcome \u2013 although, as one might expect, remote code execution vulnerabilities were more likely to be exploited compared to, say, local memory corruption bugs.<\/p>\n

EPSS was originally implemented in a spreadsheet. It provided an estimate of probability that a given vulnerability would be exploited within the next 12 months. Subsequent updates to EPSS<\/a> adopted a centralized architecture with a more sophisticated machine learning model, expanded the feature set (including variables such as public vulnerability lists, Twitter \/ X mentions, incorporation into offensive security tools, correlation of exploitation activity to vendor market share and install base, and the age of the vulnerability), and estimated the probability of exploitation within a 30-day window rather than 12 months.<\/p>\n

\"\"<\/a><\/p>\n

Figure 1: A screenshot from the EPSS Data and Statistics page, showing the top EPSS scores from the last 48 hours at the time the image was captured. Note that EPSS doesn\u2019t conclude that many of these CVEs will end up being exploited<\/em><\/p>\n

While a simple online calculator<\/a> is available for v1.0, using the latest version requires either downloading a daily CSV file from the EPSS Data and Statistics page<\/a>, or using the API<\/a>. EPSS scores are not shown on the National Vulnerability Database (NVD), which favors CVSS scores, but they are available on other vulnerability databases such as VulnDB<\/a>.<\/p>\n

As noted in our previous article in this series<\/a>, CVSS scores have not historically been a reliable predictor of exploitation, so EPSS, in principle, seems like a natural complement — it tells you about the probability <\/em>of exploitation, whereas CVSS tells you something about the impact<\/em>. As an example, say there\u2019s a bug with a CVSS Base score of 9.8, but an EPSS score of 0.8% (i.e., while severe if it is exploited, the bug is less than 1% likely to be exploited within the next 30 days). On the other hand, another bug might have a much lower CVSS Base score of 6.3, but an EPSS score of 89.9% – in which case, you might want to prioritize it.<\/p>\n

What you shouldn\u2019t do (as the EPSS authors point out) is multiply CVSS scores by EPSS scores. Even though this theoretically gives you a severity * threat value, remember that a CVSS score is an ordinal ranking. EPSS, its creators say, communicates different information from that of CVSS, and the two should be considered together but separately.<\/p>\n

So is EPSS the perfect companion to CVSS? Possibly \u2013 like CVSS, it\u2019s free to use, and offers useful insight, but it does come with some caveats.<\/p>\n

What does EPSS actually measure?<\/h2>\n

EPSS provides a probability score which indicates the likelihood of a given vulnerability being exploited in general. It does not, and is not intended to, measure the likelihood of your organization being targeted specifically, or the impact of successful exploitation, or any incorporation of an exploit into (for instance) a worm or a ransomware gang\u2019s toolkit. The outcome it predicts is binary (exploitation either occurs or it doesn\u2019t \u2013 although note that it\u2019s actually more nuanced than that: either exploitation occurs or<\/em> we don\u2019t know if it has occurred<\/em>), and so an EPSS score tells you one thing: the probability of exploitation occurring within the next 30 days. On a related note, it\u2019s worth making a note of that time period. EPSS scores should, by design, be recalculated, as they rely on temporal data. A single EPSS score is a snapshot in time, not an immutable metric.<\/p>\n

EPSS is a \u2018pre-threat\u2019 tool<\/h2>\n

EPSS is a predictive, proactive system. For any given CVE, assuming the requisite information is available, it will generate a probability that the associated vulnerability will be exploited in the next 30 days. You can then, if you choose to, factor in this probability for prioritization, provided the vulnerability has not already been exploited. <\/em>That is, the system does not provide any meaningful insight if a vulnerability is being actively exploited, because it\u2019s a predictive measure. To go back to our earlier example of logistic regression, there\u2019s little point running your data through my model and trying to sell you my product if you already bought it six weeks ago. This seems obvious, but it\u2019s still worth bearing in mind: for vulnerabilities which have been exploited, EPSS scores cannot add any value to prioritization decisions.<\/p>\n

Lack of transparency<\/h2>\n

EPSS has a similar issue to CVSS with regard to transparency, although for a different reason. EPSS is a machine learning model, and the underlying code and data is not available to most members of the FIRST SIG<\/a>, let alone the general public. While the maintainers of EPSS say that \u201cimproving transparency is one of our goals,<\/a>\u201d they also note that they cannot share data because \u201cwe have several commercial partners who requested that we not share as part of the data agreement. As far as the model and code, there are many complicated aspects to the infrastructure in place to support EPSS.\u201d<\/p>\n

Assumptions and constraints<\/h2>\n

Jonathan Spring, a researcher at Carnegie Mellon University\u2019s Software Engineering Institute, points out<\/a> that EPSS relies on some assumptions which make it less universally applicable than it may appear. EPSS\u2019s website claims that the system estimates \u201cthe likelihood (probability) that a software vulnerability will be exploited in the wild.\u201d However, there are some generalizations here. For example, \u201csoftware vulnerability\u201d refers to a published CVE \u2013 but some software vendors or bug bounty administrators might not use CVEs for prioritization at all. As Spring notes, this may be because a CVE has yet to be published for a particular issue (i.e., a vendor is coordinating with a researcher on a fix, prior to publication), or because the vulnerability is more of a misconfiguration issue, which wouldn\u2019t receive a CVE in any case.<\/p>\n

Likewise, \u201cexploited\u201d means exploitation attempts that EPSS and its partners were able to observe and record<\/a>, and \u201cin the wild\u201d means the extent of their coverage. The authors of the linked paper also note that, because much of that coverage relies on IDS signatures, there is a bias towards network-based attacks against perimeter devices.<\/p>\n

Numerical outputs<\/h2>\n

As with CVSS, EPSS produces a numerical output. And, as with CVSS, users should be aware that risk is not reducible to a single numerical score. The same applies to any attempt to combine CVSS and EPSS scores. Instead, users should take numerical scores into account while maintaining an awareness of context and the systems\u2019 caveats, which should impact how they interpret those scores. And, as with CVSS, EPSS scores are standalone numbers; there are no recommendations or interpretation guidance provided.<\/p>\n

Possible future disadvantages<\/h2>\n

The authors of EPSS note that attackers may adapt to the system<\/a>. For instance, a threat actor may incorporate lower-scoring vulnerabilities into their arsenal, knowing that some organizations may be less likely to prioritize those vulnerabilities. Given that EPSS uses machine learning, the authors also point out that attackers may in the future attempt to perform adversarial manipulation of EPSS scores, by manipulating input data (such as social media mentions or GitHub repositories) to cause overscoring of certain vulnerabilities.<\/p>\n

Stakeholder-specific Vulnerability Categorization (SSVC)<\/h1>\n

SSVC<\/a>, created by Carnegie Mellon University’s Software Engineering Institute (SEI) in collaboration with CISA in 2019, is very dissimilar to CVSS and EPSS in that it does not produce a numerical score as its output at all. Instead, it\u2019s a decision-tree model (in the traditional, logical sense, rather than in a machine learning sense). It aims to fill what its developers see as two major issues with CVSS and EPSS: a) users are not provided with any recommendations or decision points, but are expected to interpret numerical scores themselves; and b) CVSS and EPSS place the vulnerability, rather than the stakeholder, at the center of the equation.<\/p>\n

As per the SSVC whitepaper<\/a>, the framework is intended to enable decisions about prioritization, by following a decision tree along several branches. From a vulnerability management perspective, for example, you start by answering a question about exploitation: whether there\u2019s no activity, a proof-of-concept, or evidence of active exploitation. This leads to decisions about exposure (small, controlled, or open), whether the kill chain is automatable, and \u2018value density\u2019 (the resources that a threat actor would obtain after successful exploitation). Finally, there are two questions on safety impact and mission impact. The \u2018leaves\u2019 of the tree are four possible decision outcomes: defer, scheduled, out-of-cycle,<\/em> or immediate<\/em>.<\/p>\n

\"\"<\/a><\/p>\n

Figure 2: A sample decision tree from the <\/em>SSVC demo site<\/em><\/a><\/p>\n

Usefully, the latest version of SSVC also includes several other roles, including patch suppliers, coordinators, and triage\/publish roles (for decisions about triaging and publishing new vulnerabilities), and in these cases the questions and decision outcomes are different. For instance, with coordination triage, the possible outcomes are decline, track,<\/em> and coordinate<\/em>. The labels and weightings are also designed to be customizable<\/a> depending on an organization\u2019s priorities and sector.<\/p>\n

Having gone through the decision tree, you can export a result to either JSON or PDF. The result also includes a vector string, which will be familiar to anyone who read our analysis of CVSS in the previous article. Notably, this vector string contains a timestamp; some SSVC results are intended to be recalculated, depending on the context. The authors of the SSVC whitepaper recommend recalculating scores which depend on the \u2018state of exploitation\u2019 decision point once a day, for example, because this can change rapidly \u2013 whereas other decision points, such as technical impact, should be static.<\/p>\n

As the name suggests, SSVC attempts to put stakeholders at the center of the decision by emphasizing stakeholder-specific issues and decision-based outcomes, rather than numerical scores. One useful outcome of this is that you can apply the framework to vulnerabilities without a CVE, or to misconfigurations; another is that stakeholders from disparate sectors and industries can adapt the framework to suit their own needs. It\u2019s also fairly simple to use (you can try it out here<\/a>), once you\u2019ve got a handle on the definitions.<\/p>\n

To our knowledge, there hasn\u2019t been any independent empirical research into the effectiveness of SSVC, only a small pilot study conducted by SSVC\u2019s creators. The framework also prefers simplicity over nuance in some respects. CVSS, for example, has a metric for Attack Complexity, but SSVC has no equivalent decision point for ease or frequency of exploitation or anything similar; the decision point is simply whether or not exploitation has occurred and if a proof-of-concept exists.<\/p>\n

And, presumably to avoid over-complicating the decision tree, none of the decision points in any of the SSVC trees have an \u2018unknown\u2019 option by default; instead, users are advised<\/a> to make a \u201creasonable assumption\u201d based on prior events. In certain cases, this may skew the eventual decision, particularly with regards to decision points outside an organization\u2019s control (such as whether a vulnerability is being actively exploited); analysts may be uncomfortable with \u2018guessing\u2019 and err on the side of caution.<\/p>\n

That being said, it\u2019s perhaps no bad thing that SSVC avoids numerical scores (although some users may see this as a downside), and it has several other factors in its favor: It\u2019s designed to be customizable; is fully open-source; and provides clear recommendations as a final output. As with most of the tools and frameworks we discuss here, a solid approach would be to combine it with others; inputting EPSS and CVSS details (and the KEV Catalog, discussed below), where applicable, into a tailored SSVC decision tree is likely to give you a reasonable indication of which vulnerabilities to prioritize.<\/p>\n

Known Exploited Vulnerabilities (KEV) Catalog<\/h1>\n

The KEV Catalog<\/a>, operated by the Cybersecurity and Infrastructure Security Agency (CISA), is a continually updated list of which CVEs threat actors are known to have actively exploited. As of December 2024, there are 1238 vulnerabilities on that list, with provided details including CVE-ID, vendor, product, a short description, an action to be taken (and a due date, which we\u2019ll come to shortly), and a notes field, often containing a link to a vendor advisory.<\/p>\n

As per CISA\u2019s Binding Operational Directive 22-01<\/a>, \u201cfederal, executive branch, departments and agencies\u201d are required <\/em>to remediate applicable vulnerabilities in the KEV Catalog, along with some other actions, within a certain timeframe (six months for CVE-IDs assigned before 2021, two weeks for all others). CISA\u2019s justification for creating the KEV Catalog is similar to points we made in our previous article: Only a small minority of vulnerabilities are ever exploited, and attackers do not appear to rely on severity ratings to develop and deploy exploits. Therefore, CISA argues, \u201cknown exploited vulnerabilities should be the top priority for remediation\u2026[r]ather than have agencies focus on thousands of vulnerabilities that may never be used in a real-world attack.\u201d<\/p>\n

The KEV Catalog is not updated on a scheduled basis, but within 24 hours of CISA becoming aware of a vulnerability that meets certain criteria:<\/p>\n