k-Zero Day Safety: Measuring the Security Risk of Networks against Unknown Attacks Lingyu Wang1 Sushil Jajodia2, Anoop Singhal3, and Steven Noel2 1 Concordia University 2 George Mason University 3 National Institute of Standards and Technology ESORICS 2010

Outline Introduction Related Work k-Zero Day Safety Model Algorithms for Computing k-Zero Day Safety Application and Instantiation Conclusion I’ll first motivate our study, and summarize the limitations of related work. I’ll then introduce some basic concepts such as attack graph. I’ll describe our model in three stages: how to assign individual values using CVSS, how to compose them in a static case, how to compose them in dynamic case I’ll discuss two case studies.

Outline Introduction Related Work k-Zero Day Safety Model Algorithms for Computing k-Zero Day Safety Application and Instantiation Conclusion I’ll first motivate our study, and summarize the limitations of related work. I’ll then introduce some basic concepts such as attack graph. I’ll describe our model in three stages: how to assign individual values using CVSS, how to compose them in a static case, how to compose them in dynamic case I’ll discuss two case studies.

The Need for Security Metric “Much more secure”? How much more? … … Boss, we really need this new firewall, it will make our network much more secure! Networks may contain residue vulnerabilities, the reason is as the following… To deal with such residue vulnerabilities, there exist qualitative solutions, such as…

The Need for Security Metric “You cannot improve what you cannot measure” To justify the cost of a security solution, we need to know how much security the solution can bring A security metric will allow for a direct measurement of security before and after deploying the solution Such a capability will make network hardening a science rather than an art Networks may contain residue vulnerabilities, the reason is as the following… To deal with such residue vulnerabilities, there exist qualitative solutions, such as…

The Need for Security Metric “Much more secure”? How much more? Security Cost 2 $5k 3 $10k … Networks may contain residue vulnerabilities, the reason is as the following… To deal with such residue vulnerabilities, there exist qualitative solutions, such as…

Can Security Be Measured? Security metric exists for known vulnerabilities1 Knowledge about vulnerabilities allow us to measure their relative exploitability, likelihood, impact, etc. But what about unknown vulnerabilities? We are measuring the unmeasurable2, because there is little ground for such a measurement Vulnerability: No prior knowledge is available Software: Software flaws are much less predictable Attacker: Finding flaws/developing exploits is a chaotic process 1 Common Vulnerability Scoring System (CVSS-SIG) v2, http://www.first.org/cvss/ 2 J. McHugh. Quality of protection: Measuring the unmeasurable? In Proceedings of the 2nd ACM workshop on Quality of protection (QoP’06), 2006.

The Curse on Security Metric What if we can’t measure unknown vulnerabilities? Attackers can simply step outside and do as he pleases1 What’s the value of a “more secure” system that is equally susceptible to unknown attacks? Therefore, security is not quantifiable until we can fix all potential flaws But by then we certainly don’t need a security metric! 1 J. McHugh. Quality of protection: Measuring the unmeasurable? In Proceedings of the 2nd ACM workshop on Quality of protection (QoP’06), 2006.

The Curse on Security Metric Our Solution What if we can’t measure unknown vulnerabilities? Attackers can simply step outside and do as he pleases1 What’s the value of a “more secure” system that is equally susceptible to unknown attacks? Therefore, security is not quantifiable until we can fix all potential flaws But by then we certainly don’t need a security metric! don’t Instead, we simply count them We count how many unknown vulnerabilities can be resisted by a network A larger count means a more secure network Since more unknown vulnerabilities must all be Available at the same time, Applicable to the same network, and Exploitable by the same attacker Whose likelihood is lower Those existing qualitative solutions can already answer following questions… However, all those questions have a qualitative nature… We need to quantify the security of a network

Our Contribution The k-zero day safety metric Formally defined based on an abstract network model Proved to satisfy the required algebra properties Algorithms for computing the metric are proposed Application to network hardening is discussed The first known effort capable of quantifying the risk of unknown attacks It may open up new opportunities to the evaluation, hardening, and design of secure networks CVSS measures the exploitability, with its temporal factors, of a vulnerability. The interplay between vulnerabilities in a given network is not taken into account in CVSS. The impact means the impact of an individual vulnerability, without considering the context.

Outline Introduction Related Work k-Zero Day Safety Model Algorithms for Computing k-Zero Day Safety Application and Instantiation Conclusion I’ll first motivate our study, and summarize the limitations of related work. I’ll then introduce some basic concepts such as attack graph. I’ll describe our model in three stages: how to assign individual values using CVSS, how to compose them in a static case, how to compose them in dynamic case I’ll discuss two case studies.

Related Work NIST’s efforts on standardizing security metric Special publication 500-133 1985, 800-55 2003 CVSSv2 and NVD Efforts on measuring known vulnerabilities MTTF-based approach (Dacier et al., TSE’99) Minimum-effort approaches (Balzarotti et al., QoP’05 and Pamula et al., QoP’06) PageRank approach (Mehta et al., RAID’06) Our previous work (DBSec’07-08, QoP’07-08) NIST has several efforts on security metric… Dacier and others proposed to use Markov model and mean time to failure (MTTF) to measure security, but they do not consider realistic cases modeled by attack graphs. Several previous approaches measure security using the minimum-efforts required by attackers, we have shown in last year’s QoP workshop that those approaches have their limitations. Attack surface is for software security. Pagerank assumes attacker moves in a random way, which is not necessarily the case.

Related Work Attack surface (Howard et al., QoP’06) Measures the security of a single software system Focusing on interfaces instead of internal details k-anonymity (Samarati et al., TKDE’01) Measuring the amount of privacy using an integer regardless of specific application semantic Zero day attack Total number of zero-day vulnerabilities (McQueen et al., HICSS’09) Ranking applications with consequences of having one zero-day vulnerability (Ingols et al., ACSAC’09) NIST has several efforts on security metric… Dacier and others proposed to use Markov model and mean time to failure (MTTF) to measure security, but they do not consider realistic cases modeled by attack graphs. Several previous approaches measure security using the minimum-efforts required by attackers, we have shown in last year’s QoP workshop that those approaches have their limitations. Attack surface is for software security. Pagerank assumes attacker moves in a random way, which is not necessarily the case.

Outline Introduction Related Work k-Zero Day Safety Model Algorithms for Computing k-Zero Day Safety Application and Instantiation Conclusion I’ll first motivate our study, and summarize the limitations of related work. I’ll then introduce some basic concepts such as attack graph. I’ll describe our model in three stages: how to assign individual values using CVSS, how to compose them in a static case, how to compose them in dynamic case I’ll discuss two case studies.

Network An example The model H ={0,1,2,F} S={http,ssh,iptables,firewall} P={user,root} conn={<0,F>,<0,1>…} serv(1)={http,ssh,iptables} serv(F)={firewall} priv(1)=priv(2)={user,root} E union C are the nodes, Rr union Ri are the edges. Vul is vulnerability name, source and dest are two hosts. Rr and Ri are two relations that comprise the edges of attack graphs. If all services are free of known vulnerabilities, a vulnerability scanner or attack graph will claim the network is secure, and no additional hardening effort (e.g., iptables) is necessary

Assumptions However, we shall reach a different conclusion by considering at least how many zero-day attacks are required to compromise the network We assume a zero day vulnerability Cannot be exploited unless A network connection exists between source/destination A remote service with the vulnerability exists on destination The attacker already has a privilege on the source host May lead to any privilege on the destination host (These essentially depict a worst-case scenario) NIST has several efforts on security metric… Dacier and others proposed to use Markov model and mean time to failure (MTTF) to measure security, but they do not consider realistic cases modeled by attack graphs. Several previous approaches measure security using the minimum-efforts required by attackers, we have shown in last year’s QoP workshop that those approaches have their limitations. Attack surface is for software security. Pagerank assumes attacker moves in a random way, which is not necessarily the case.

Zero Day Vulnerability An example The model <vssh,0,1>: <0,1>conn  sshserv(1) <vroot,1,1>: root priv(1) pre(<vssh,0,1>)={<0,1>,<ssh,1>,<user,0>} post(<vssh,0,1>)={<user,1>} pre(<vroot,1,1>)={<user,1>} post(<vroot,1,1>)={<root,1>} E union C are the nodes, Rr union Ri are the edges. Vul is vulnerability name, source and dest are two hosts. Rr and Ri are two relations that comprise the edges of attack graphs.

k-Zero Day Safety An example The model CI={<user,0>} A={<root,2>} <vhttp,0,1>v<vssh,1,2> <vssh,0,1>v<vssh,1,2> <vssh,0,2>v<vroot,2,2> k0d({<vhttp,0,1>,<vssh,1,2>})=2 k0d({<vssh,0,1>,<vssh,1,2>})=1 k0d({<vfirewall,0,F>,<vssh,0,2>},<vroot,2,2>)=3 E union C are the nodes, Rr union Ri are the edges. Vul is vulnerability name, source and dest are two hosts. Rr and Ri are two relations that comprise the edges of attack graphs. At least one zero day vulnerability is required to compromise the network

Hardening the Network: k=k+1 An example The model <viptables,0,1>v<vssh,0,1> <viptables,0,1>v<vssh,1,2> k0d({<viptables,0,1>,<vssh,1,2>})=2 k0d({<viptables,0,1>,<vssh,0,1>,<vssh,1,2>})=2 k0d(<root,2>)=2 E union C are the nodes, Rr union Ri are the edges. Vul is vulnerability name, source and dest are two hosts. Rr and Ri are two relations that comprise the edges of attack graphs. With this hardening effort, at least two distinct zero day vulnerabilities are required to compromise the same network

In Summary Our metric can help to compare the relative security of “secure networks” that are otherwise indistinguishable by existing techniques (Notice: Many features of the model are not mentioned while discussing this simple example. More details can be found in the paper) E union C are the nodes, Rr union Ri are the edges. Vul is vulnerability name, source and dest are two hosts. Rr and Ri are two relations that comprise the edges of attack graphs.

Outline Introduction Related Work k-Zero Day Safety Model Algorithms for Computing k-Zero Day Safety Application and Instantiation Conclusion I’ll first motivate our study, and summarize the limitations of related work. I’ll then introduce some basic concepts such as attack graph. I’ll describe our model in three stages: how to assign individual values using CVSS, how to compose them in a static case, how to compose them in dynamic case I’ll discuss two case studies.

What’s the Value of k? An example The algorithm Complexity <root,2> =<vssh,1,2><vssh,0,2> =<vssh,1,2><root,1><vssh,0,2><0,2> =… (DNF conversion) =(<vhttp,0,1><vssh,1,2>)(<vssh,0,1><vssh,1,2>)(<vfirewall,0,F><vssh,0,2>) k=k0d({<vssh,0,1>,<vssh,1,2>})=1 Complexity Exponential (in size of the attack graph) The problem is NP-hard Efficient algorithms still exist for practical variations E union C are the nodes, Rr union Ri are the edges. Vul is vulnerability name, source and dest are two hosts. Rr and Ri are two relations that comprise the edges of attack graphs.

Is k>1 True? An example The algorithm Complexity <user,0> <vhttp,0,1><vssh,1,2>) (k>1) <vssh,0,1><vssh,1,2>) (k=1) (k>1)=FALSE! Complexity Polynomial if k is compared to a constant (in size of the attack graph) E union C are the nodes, Rr union Ri are the edges. Vul is vulnerability name, source and dest are two hosts. Rr and Ri are two relations that comprise the edges of attack graphs.

Outline Introduction Related Work k-Zero Day Safety Model Algorithms for Computing k-Zero Day Safety Application and Instantiation Conclusion I’ll first motivate our study, and summarize the limitations of related work. I’ll then introduce some basic concepts such as attack graph. I’ll describe our model in three stages: how to assign individual values using CVSS, how to compose them in a static case, how to compose them in dynamic case I’ll discuss two case studies.

Application to Network Hardening This (mess) tells us (in number) that k may be increased by: Increasing diversity of services Strengthening isolation around services Removing unnecessary services or connections Enforcing stricter access control policies Protecting assets via backups or IDSs Introducing more security services Patching known vulnerabilities …… We can unfold k based on the model NIST has several efforts on security metric… Dacier and others proposed to use Markov model and mean time to failure (MTTF) to measure security, but they do not consider realistic cases modeled by attack graphs. Several previous approaches measure security using the minimum-efforts required by attackers, we have shown in last year’s QoP workshop that those approaches have their limitations. Attack surface is for software security. Pagerank assumes attacker moves in a random way, which is not necessarily the case.

Application to Network Hardening Nothing new here? Right, these hardening options match existing practices (e.g., layered defense, security via virtualization, security through diversity, etc.) Which shows the relevance of our metric But their effectiveness can now be quantified! And their cost can be justified In a simple, intuitive way (so simple that even the boss can understand) We can unfold k based on the model k Cost 2 $5k 3 $10k … NIST has several efforts on security metric… Dacier and others proposed to use Markov model and mean time to failure (MTTF) to measure security, but they do not consider realistic cases modeled by attack graphs. Several previous approaches measure security using the minimum-efforts required by attackers, we have shown in last year’s QoP workshop that those approaches have their limitations. Attack surface is for software security. Pagerank assumes attacker moves in a random way, which is not necessarily the case.

Instantiating the Model k=3 This paper focuses on model and algorithms Instantiating the model from a real world network is a different issue We discuss several key aspects in the paper Algorithms Model NIST has several efforts on security metric… Dacier and others proposed to use Markov model and mean time to failure (MTTF) to measure security, but they do not consider realistic cases modeled by attack graphs. Several previous approaches measure security using the minimum-efforts required by attackers, we have shown in last year’s QoP workshop that those approaches have their limitations. Attack surface is for software security. Pagerank assumes attacker moves in a random way, which is not necessarily the case. Instantiation

Outline Introduction Related Work k-Zero Day Safety Model Algorithms for Computing k-Zero Day Safety Application and Instantiation Conclusion I’ll first motivate our study, and summarize the limitations of related work. I’ll then introduce some basic concepts such as attack graph. I’ll describe our model in three stages: how to assign individual values using CVSS, how to compose them in a static case, how to compose them in dynamic case I’ll discuss two case studies.

Conclusion We have We can unfold k based on the model proposed the k-zero day safety metric discussed algorithms and complexity shown potential application of the metric Future work include extending the model to address various limitations further investigating instantiation of the model studying other applications of the metric We can unfold k based on the model NIST has several efforts on security metric… Dacier and others proposed to use Markov model and mean time to failure (MTTF) to measure security, but they do not consider realistic cases modeled by attack graphs. Several previous approaches measure security using the minimum-efforts required by attackers, we have shown in last year’s QoP workshop that those approaches have their limitations. Attack surface is for software security. Pagerank assumes attacker moves in a random way, which is not necessarily the case.

Q & A Thank You! Contact Author: Lingyu Wang (wang@ciise.concordia.ca)