A Mission-Centric Framework for Cyber Situational Awareness Metrics, Lifecycle of Situational Awareness, and Impact of Automated Tools on Analyst Performance S. Jajodia, M. Albanese George Mason University ARO-MURI on Cyber-Situation Awareness Review Meeting Santa Barbara, CA , November 18-19, 2014

Outline Overview of Mason’s Role Year 5 Statistics Metrics Measuring Security Risk Network Diversity Lifecycle of Situational Awareness Impact of SA on Analyst Performance Conclusions ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Overview of Mason’s Role ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Where We Stand in the Project Cognitive Models & Decision Aids Instance Based Learning Models Simulation Measures of SA & Shared SA Software Sensors, probes Hyper Sentry Cruiser Data Conditioning Association & Correlation Information Aggregation & Fusion Transaction Graph methods Damage assessment Automated Reasoning Tools R-CAST Plan-based narratives Graphical models Uncertainty analysis Multi-Sensory Human Computer Interaction Computer network Enterprise Model Activity Logs IDS reports Vulnerabilities Real World System Analysts Test-bed Computer network ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Vulnerability Databases Our Vision Vulnerability Databases NVD OSVD CVE Scenario Analysis & Visualization Analyst Zero-day Analysis Network Hardening Unexplained Behavior Analysis Cauldron Topological Vulnerability Analysis Index & Data Structures Graph Processing and Indexing Cauldron Switchwall Stochastic Attack Models Situation Knowledge Reference Model [Attack Scenario Graphs] Monitored Network Dependency Analysis NSDMiner Generalized Dependency Graphs Alerts/Sensory Data ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Overview of Contribution – Year 1 Technical accomplishments A topological approach to Vulnerability Analysis that overcomes the drawbacks of traditional point-wise vulnerability analysis Preliminary data structures and graph-based techniques and algorithms for processing alerts/sensory data A novel security metric, k-zero day safety, to assess how many zero-day vulnerabilities are required for compromising a network asset Major breakthroughs Capability of processing massive amounts of alerts in real-time Capability of forecasting possible futures of the current situation Capability of hardening a network against zero day vulnerabilities ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Overview of Contribution – Year 2 Technical accomplishments Generalized dependency graphs, which capture how network components depend on one other Probabilistic temporal attack graphs, which encode probabilistic and temporal knowledge of the attacker’s behavior Attack scenario graphs, which combine dependency and attack graphs Efficient algorithms for both detection and prediction A preliminary model to identify “unexplained” cyber activities, i.e., activities incompatible with any given known activity model Major breakthroughs Capability of generating and ranking future attack scenarios in real time ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Overview of Contribution – Year 3 Technical accomplishments An efficient and cost-effective algorithm to harden a network with respect to given security goals A probabilistic framework for localizing attackers in mobile networks A probabilistic framework for assessing the completeness and quality of available attack models (joint work with UMD and ARL) A suite of novel techniques to automatically discover dependencies between network services from passively collected network traffic Switchwall, an Ethernet-based network fingerprinting technique for detecting unauthorized changes to the L2/L3 network topology Major breakthroughs Capability of automatically and efficiently executing several important analysis tasks, namely hardening, dependency analysis, and attacker localization ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Overview of Contribution – Year 4 Technical accomplishments Effective and efficient methods for generating partial attack graphs on demand in order to enable efficient analysis of zero-day vulnerabilities A three-step process to assess the risk associated with zero-day vulnerabilities A prototype of the probabilistic framework for unexplained activity analysis Major breakthroughs Capability to reason about zero-day vulnerabilities and efficiently assess the risk associated with such vulnerabilities without generating the entire attack graph ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Overview of Contribution – Year 5 Technical accomplishments A suite of metrics for measuring network-wide cyber security risk based on attack graphs An approach to model network diversity as a security metric for evaluating the robustness of networks against zero-day attacks An analysis of how situational awareness forms and evolves during the several stages of the cyber defense process An analysis of how automated CSA tools can be used for improving analyst performance Major breakthroughs Capability of quantifying risk and resiliency using several metrics ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Quad Chart - Year 5 Objectives: DoD Benefit: Major Accomplishments Improve Cyber Situation Awareness via Metrics for measuring network-wide cyber security risk An better understanding of the impact of network diversity on the robustness of networks against zero-day attacks A better understanding of how situational awareness forms and evolves A better understanding of how automated CSA tools can improve analyst performance DoD Benefit: Ability to quantitatively evaluate network-wide security risks Ability to better design automated CSA tools that can effectively reduce the workload for the analysts and improve their performance Scientific/Technical Approach Defining a hierarchy of attack graph based metrics, and developing metrics Studying diversity as a network-wide metrics to asses resilience against zero-day attacks, and defining several diversity-based metrics: biodiversity inspired, least attacking effort, and average attacking effort Studying situational awareness capabilities from a functional point of view, and identifying inputs, outputs, and lifecycle of the derived awareness Examining the impact of automated tools on analyst performance Major Accomplishments Defined a suite of metrics for measuring network-wide cyber security risk based on a model of multi-step attack vulnerability (attack graph) Modeled network diversity as a security metric for evaluating the robustness of networks against zero-day attacks Studied how situational awareness forms and evolves during the several stages of the cyber defense process, and how automated CSA tools can be used for improving analyst performance Challenges Defining solid metrics that accurately capture risk and resilience ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Year 5 Statistics ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Year 5 Statistics (1/2) Publications & presentations 3 papers published in peer-reviewed conference proceedings 1 paper published in a peer-reviewed journal 2 book chapters 1 book L. Wang, M. Albanese, and S. Jajodia, “Network Hardening: An Automated Approach to Improving Network Security,” ISBN 978-3- 319-04611-2, SpringerBriefs in Computer Science, 2014, 60 pages Supported personnel 2 faculty 1 doctoral student 1 undergraduate student ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Year 5 Statistics (2/2) Patents Awarded during the reporting period Sushil Jajodia, Lingyu Wang, and Anoop Singhal, “Interactive Analysis of Attack Graphs Using Relational Queries”, United States Patent No. 8,566,269 B2, October 22, 2013. Steven Noel, Sushil Jajodia, and Eric Robertson, “Intrusion Event Correlation System”, United States Patent No. 8,719,943 B2, May 6, 2014. Patents Disclosed during the reporting period Massimiliano Albanese, Sushil Jajodia, and Steven Noel, “Methods and Systems for Determining Hardening Strategies”, United States Patent Application No. US 2014/0173740 A1, June 19, 2014. Honors & Awards Max Albanese received the 2014 Mason Emerging Researcher/Scholar/Creator Award ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Metrics: Measuring Security Risk Steven Noel and Sushil Jajodia, “Metrics suite for network attack graph analytics,” Proceedings of the 9th Cyber and Information Security Research Conference (CISR 2014), Oak Ridge, TN, USA, April 8-10, 2014 ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Overview Attack (vulnerability dependency) graphs Attack graph metrics Combine information about topology, policy, and vulnerabilities Identify network vulnerability paths Provide qualitative rather than quantitative insights Attack graph metrics Capture trends over time Enable comparisons across organizations Look at complementary dimensions of security ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Cauldron Attack Graph ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Attack Graph Metrics Metrics Engine Metrics Dashboard Network Topology Analysis XML CSV Graphical Metrics Engine Firewall Rules Cisco ASA Cisco IOS Juniper JUNOS Juniper ScreenOS … Host Vulnerabilities Nessus Retina nCircle nmap … Metrics Dashboard ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Attack Graph Metrics Families Victimization: Individual vulnerabilities and exposed services each have elements of risk We score the entire network across individual vulnerability victimization dimensions Size: The size of attack graphs is a prime indication of risk The larger the graph, the more ways to be compromised Containment: Networks are generally administered in pieces (subnets, domains, etc.) Risk mitigation should aim to reduce attacks across such boundaries Topology: The connectivity, cycles, and depth of the attack graph indicate how graph relationships enable network penetration ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Metrics Hierarchy Overall Victimization Existence Exploitability Impact Size Vectors Machines Containment Vuln Types Topology Connectivity Cycles Depth Network Score Metrics Family Individual Metrics ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Victimization Metrics Existence – relative number of ports that are vulnerable (on a 0 to 10 scale) Exploitability – average CVSS Exploitability Impact – average CVSS Impact ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Size Family: Vectors Metric Across domains: explicit vectors Within domain (implicit vectors) ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Size Family: Machines Metric Non-vulnerable machines Vulnerable machines ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Containment Family: Vectors Metric Across domains: explicit vectors Within domain (implicit vectors) ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Containment Family: Machines Metric Victims within domain only Victims across domains ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Containment Family: Vulnerability Types within domain only Vulnerability types across domains ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Attack Graph Connectivity Motivation: Better to have attack graph as disconnected parts versus connected whole One Component Two Components Three Components Less Secure More ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Topology Family: Connectivity Metric 1 component 4 components 5 components ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Attack Graph Cycles More Less Secure Motivation: For a connected attack graph, better to avoid cycles among subgraphs Less Secure More ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Topology Family: Cycles Metric 4 components 5 components 10 components ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Motivation: Better to have attack graph deeper versus shallower Attack Graph Depth Motivation: Better to have attack graph deeper versus shallower One Step Deep 2 Steps Deep 3 Steps Deep Less Secure More ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Topology Family: Depth Metric Shortest path 3/8 Shortest path 4/8 Shortest paths 2/3 and 1/5 ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Metrics Dashboard ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Trend Summary ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Example Network Topology Partner Domains DMZ Internal Domains ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Attack Graph – Before Hardening ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Attack Graph – After Hardening ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Metrics: Network Diversity L. Wang, M. Zhang, S. Jajodia, A. Singhal, and M. Albanese, “Modeling Network Diversity for Evaluating the Robustness of Networks against Zero-Day Attacks,” Proceedings of the 19th European Symposium on Research in Computer Security (ESORICS 2012), Wroclaw, Poland, September 7-11, 2014 ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Overview Zero-day attacks are a real threat to mission critical networks Governments and cybercriminals are stockpiling zero-day vulnerabilities1 The NSA spent more than $25 million a year to acquire software vulnerabilities Example. Stuxnet exploits 4 different/complementary zero day vulnerabilities to infiltrate a SCADA network But what can we do about unknown attacks? 1 http://krebsonsecurity.com/2013/12/how-many-zero-days-hit-you-today/ ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

How Could Diversity Help? Stuxnet’s attack strategy 3rd party (e.g., contractor)  organization’s network  machine with Siemens Step 7  PLC The degree of software diversity along potential attack paths can be considered a good metric for the network’s capability of resisting Stuxnet ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Existing Work on Diversity Software diversity has long been regarded as a security mechanism for improving robustness The degree of diversity along potential attack paths is an indicator of the network’s capability of resisting attacks Tolerating attacks as Byzantine faults by comparing outputs or behaviors of diverse variants Limitations: At a higher abstraction level, as a global property of an entire network, network diversity and its impact on security has not been formally modeled ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Our Contribution We take the first step towards formally modeling network diversity as a security metric We propose a network diversity function based on well known mathematical models of biodiversity in ecology We design a network diversity metric based on the least attacking effort We design a probabilistic network diversity metric to reflect the average attacking effort We evaluate the metrics and algorithms through simulation The modeling effort helps understand diversity and enables quantitative hardening approaches CVSS measures the exploitability, with its temporal factors, of a vulnerability. The interplay between vulnerabilities in a given network is not taken into account in CVSS. The impact means the impact of an individual vulnerability, without considering the context. ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Bio-Diversity and Richness of Species Literature on biodiversity confirms a positive relationship between biodiversity and the ecosystem’s resistance to invasion and diseases Richness of species The number of different species in an ecosystem Limitation: ignores the relative abundance of each species Effective number or resources Measures the equivalent number of equally-common species, even if in reality all species are not equally common Limitation: assumes all resources are equally different Similarity-Sensitive Effective Richness We can use a resource similarity function to account for differences between resources NIST has several efforts on security metric… Dacier and others proposed to use Markov model and mean time to failure (MTTF) to measure security, but they do not consider realistic cases modeled by attack graphs. Several previous approaches measure security using the minimum-efforts required by attackers, we have shown in last year’s QoP workshop that those approaches have their limitations. Attack surface is for software security. Pagerank assumes attacker moves in a random way, which is not necessarily the case. ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Resource Graph Syntactically equivalent to an attack graph Models causal relationships between network resources (rather than vulnerabilities) Vertices: zero-day exploits, their pre- and post-conditions Edges: AND between pre-conditions, OR between exploits On which path should we compute the diversity metrics? NIST has several efforts on security metric… Dacier and others proposed to use Markov model and mean time to failure (MTTF) to measure security, but they do not consider realistic cases modeled by attack graphs. Several previous approaches measure security using the minimum-efforts required by attackers, we have shown in last year’s QoP workshop that those approaches have their limitations. Attack surface is for software security. Pagerank assumes attacker moves in a random way, which is not necessarily the case. ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Selecting the Least Diverse Path(s) Intuitively, it should be the “shortest” path 1 or 2 have the minimum number of steps, but 4 may take less effort than 1! 2 or 4 have the minimum number of resources? But they both have 2 resources, so which one is better? 4 minimizes #resources/#steps? But what if there is a path with 9 steps and 3 resources? 1/3<2/4, but it clearly does not represent the least attack effort! NIST has several efforts on security metric… Dacier and others proposed to use Markov model and mean time to failure (MTTF) to measure security, but they do not consider realistic cases modeled by attack graphs. Several previous approaches measure security using the minimum-efforts required by attackers, we have shown in last year’s QoP workshop that those approaches have their limitations. Attack surface is for software security. Pagerank assumes attacker moves in a random way, which is not necessarily the case. ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Network Diversity in Least Attack Effort We define network diversity as: 𝑚𝑖𝑛𝑖𝑚𝑢𝑚 # 𝑜𝑓 𝑟𝑒𝑠𝑜𝑢𝑟𝑐𝑒𝑠 𝑜𝑛 𝑎𝑛𝑦 𝑝𝑎𝑡ℎ 𝑚𝑖𝑛𝑖𝑚𝑢𝑚 # 𝑜𝑓 𝑠𝑡𝑒𝑝𝑠 𝑜𝑛 𝑎𝑛𝑦 𝑝𝑎𝑡ℎ Note: These may or may not be the same path! In this case: 2 (path 2, 4) / 3 (path 1, 2) Determining the network diversity is NP-hard Our heuristic algorithm only keeps a limited number of local optima at each step NIST has several efforts on security metric… Dacier and others proposed to use Markov model and mean time to failure (MTTF) to measure security, but they do not consider realistic cases modeled by attack graphs. Several previous approaches measure security using the minimum-efforts required by attackers, we have shown in last year’s QoP workshop that those approaches have their limitations. Attack surface is for software security. Pagerank assumes attacker moves in a random way, which is not necessarily the case. ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Network Diversity in Average Effort The least attacking effort-based metric only provides a partial picture of the threat We now define a probabilistic network diversity metric based on the average attacking effort Defined as 𝑝 1 𝑝 2 , where 𝑝 1 is the probability an attacker can compromise a given asset now, and 𝑝 2 is the probability he/she can still compromise it if all the resources were to be made different (i.e., every resource type would appear at most once) NIST has several efforts on security metric… Dacier and others proposed to use Markov model and mean time to failure (MTTF) to measure security, but they do not consider realistic cases modeled by attack graphs. Several previous approaches measure security using the minimum-efforts required by attackers, we have shown in last year’s QoP workshop that those approaches have their limitations. Attack surface is for software security. Pagerank assumes attacker moves in a random way, which is not necessarily the case. ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Simulation Results Accuracy and Performance ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Lifecycle of Situational Awareness M. Albanese and S. Jajodia, “Formation of Awareness,” to appear in Cyber Defense and Situational Awareness, A. Kott, R. Erbacher, C. Wang, eds., Springer Advances in Information Security, 2014. ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Cyber Defense Process at a Glance The overall process of cyber defense relies on the combined knowledge of actual attacks and effective defenses It ideally involves every part of the ecosystem The enterprise, its employees and customers, and other stakeholders It also entails the participation of individuals in every role within the organization Threat responders, security analysts, technologists, tool developers, users, policymakers, auditors, etc. Defensive actions are not limited to preventing the initial compromise They also address detection of already-compromised machines and prevention or disruption of attackers’ subsequent actions The defenses identified deal with reducing the initial attack surface Hardening device configurations, addressing long-term threats (such as APTs), disrupting attackers’ command-and-control of implanted malicious code, and establishing an adaptive defense and response capability ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Cyber Defense Critical Functions Learning from attacks Using knowledge of actual attacks that have compromised a system to provide the foundation to learn from these events and build effective, practical defenses Prioritization Prioritizing controls that will provide the greatest risk reduction and protection against current and future threats Metrics Establishing common metrics to provide a shared language for all parties involved to measure the effectiveness of security controls Continuous diagnostics and mitigation Carrying out continuous measurement to test and validate the effectiveness of current security controls, and to help drive the prioritization of the next steps Automation Automating defenses so that organizations can achieve reliable, scalable, and continuous monitoring of security relevant events and variables ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Cyber Defense Roles Security Analyst Security Engineer Responsible for analyzing and assessing existing vulnerabilities in the IT infrastructure, and investigating available tools and countermeasures Security Engineer Responsible for performing security monitoring, detecting security incidents, and initiating incident response Security Architect Responsible for designing a security system or its major components Security Administrator Responsible for managing organization-wide security systems Security consultant/specialist Responsible for different task related to protecting computers, networks, software, data, and/or information systems against cyber threats ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Questions ARO-MURI on Cyber-Situation Awareness Review Meeting Current situation. Is there any ongoing attack? If yes, where is the attacker? Impact. How is the attack impacting the enterprise or mission? Can we asses the damage? Evolution. How is the situation evolving? Can we track all the steps of an attack? Behavior. How are the attackers expected to behave? What are their strategies? Internet Web Server (A) Mobile App Server (C) Catalog Server (E) Order Processing Server (F) DB Server (G) Local DB Server (D) Local DB Server (B) Forensics. How did the attacker create the current situation? What was he trying to achieve? Prediction. Can we predict plausible futures of the current situation? Information. What information sources can we rely upon? Can we assess their quality? Scalability. How can we ensure that solutions scale well for large networks? ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

1 – Current Situation Is there any ongoing attack? If yes, what is the stage of the intrusion and where is the attacker? Capability Effectively detecting ongoing intrusions, and identifying the assets that might have been compromised already Input IDS logs, firewall logs, and data from other security monitoring tools Output A detailed mapping of current intrusive activities Lifecycle This type of SA may quickly become obsolete – if not updated frequently – as the intruder progresses within the system ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

2 – Impact How is the attack impacting the organization or mission? Can we assess the damage? Capability Accurately assessing the impact (so far) of ongoing attacks Input Knowledge of the organization’s assets along with some measure of each asset’s value Output An estimate of the damage caused so far by the intrusive activity Lifecycle This type of SA must be frequently updated to remain useful, as damage will increase as the attack progresses ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

3 – Evolution How is the situation evolving? Can we track all the steps of an attack? Capability Monitoring ongoing attacks, once such attacks have been detected Input Situational awareness generated in response to the questions 1 &2 Output A detailed understanding of how the attack is progressing Lifecycle This capability can help address the limitations on the useful life of the situational awareness generated in response to questions 1 & 2 ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

How are the attackers expected to behave? What are their strategies? 4 – Behavior How are the attackers expected to behave? What are their strategies? Capability Modeling the attacker’s behavior in order to understand its goals and strategies Input Past observations and knowledge of organization’s assets Output A set of formal models (e.g., game theoretic, stochastic) of the attacker’s behavior Lifecycle The attacker’s behavior may change over time, therefore models need to adapt to a changing adversarial landscape ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

5 – Forensics How did the attacker create the current situation? What was he trying to achieve? Capability Analyzing the logs after the fact and correlating observations in order to understand how an attack originated and evolved Input Situational awareness gained is response to question 4 Output A detailed understanding of the weaknesses and vulnerabilities that made the attack possible Lifecycle This information can help security engineers and administrators harden system configurations to prevent similar incidents from happening again ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Can we predict plausible futures of the current situation? 6 – Prediction Can we predict plausible futures of the current situation? Capability Predicting possible moves an attacker may take in the future Input Situational awareness gained in response to questions 1, 3, and 4 Output A set of possible alternative scenarios that may realize in the future Lifecycle This type of SA may quickly become obsolete ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

7 – Quality of Information What information sources can we rely upon? Can we assess their quality? Capability Assessing the quality of the information sources all other tasks depend upon Input Information sources Output A detailed understanding of how to weight different sources when processing information in response to other questions Lifecycle Needs to be updated when the information sources change ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Impact of SA on Analyst Performance M. Albanese, H. Cam, and S. Jajodia, “Automated Cyber Situation Awareness Tools for Improving Analyst Performance,” Cybersecurity Systems for Human Cognition Augmentation, Springer 2014. ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Overview Automated Cyber Situation Awareness tools and models can enhance performance, cognition and understanding for cyber professionals monitoring complex cyber systems In most current solutions, human analysts are heavily involved in every phase of the monitoring and response process Ideally, we should move from a human-in-the loop scenario to a human-on-the loop scenario Human analysts should have the responsibility to oversee the automated processes and validate the results of automated analysis of monitoring data To this aim, it is highly desirable to have temporal models such as Petri nets to model and integrate the concurrent operations of cyber-physical systems with the cognitive processing of analyst ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Petri Net Models for SA ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Conclusions ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Conclusions The focus in Year 5 was on integration of previous contributions refinement of the CSA framework definition of metrics attack graph based diversity based better understanding the overall process lifecycle of CSA role of the analyst Some of these capabilities will be further refined in a side project ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014

Questions? ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014