1. The Roles of Intrusion Detection and Data Fusion in Cyber Security Situational Awareness A Review of the Published Literature and Discussion of Future Research Plans Nicklaus A. Giacobe

2. Intrusion Detection (ID) Plays and Important Role in Developing Situational Awareness Cyber Situational Awareness = Network Security Situational Awareness Activities Performed on Behalf of an Organization – “Network Security Office” Activities Performed by Computer/Network Security Analysts Difficult, Complex Work – Lots of Data from IDS, Antivirus Systems, Firewall Logs, Server Security Logs, etc. Ever-Changing Landscape - New Threats, New Technologies, New Software, New Vulnerabilities Cyber Security Situational Awareness Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

3. This Introduction Part 1: What is the Current State of ID Technology? Part 2: What are We Trying to Accomplish? Part 3: Future Research Recommendations Conclusion/Discussion Cyber Security Situational Awareness Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

4. History of ID Alert Correlation and Data Fusion Data Fusion Techniques Visualizations Part 1: The Current State of Technology in ID Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

5. History of ID Alert Correlation and Data Fusion Data Fusion Techniques Visualizations Part 1: The Current State of Technology in ID Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

6. Two Different Locations to Monitor Host-Based IDS (Denning) Log Files (C2 compliance) on Unix Machines (Denning 1987) IDES/NIDES – Baseline “normal” user behavior (Javitz et al. 1994) Network-Based IDS (Mukherjee/Heberlein) NSM (LAN Monitor) – history of previous connections, known bad actors lists, signatures of attack types (Mukherjee et al. 1994) NIDS (Multiple Network IDS and Host) (Snapp et al, 1991) (interesting JDL comparison) History of Intrusion Detection Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

7. Two Different Methods of Analysis Pattern-Matching (Misuse) Detection (Spafford) Match activity to patterns of known undesired behavior (Kumar et al. 1994, 1995) Tripwire – MD Hashing of files (Kim et al. 1994) DDoS prevention /SYN Floods / Active DoS prevention (Schuba et al. 1997) Anomaly Detection (Stolfo) Looking for abnormalities in network traffic (Lee et al. 1999) Qualitative evaluation of the data stream (statistical methods) (Portnoy, et al. 2001) – alert on infrequent types of data Statistical Payload Evaluations – for Worm Detection (Wang et al. 2004, 2006a, 2006b) and mitigation (Locasto et al., 2006) History of Intrusion Detection Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

8. Testing and Evaluation of IDSs DARPA IDS Data Sets from 1998-2000 1999 Data Set Contained 2 Weeks of “training data” with labeled known intrusions 7 Weeks of unlabeled data Evaluate IDSs under design or in production Over-fit problem IDSs could be developed that find all of the problems in the “training data”, but could be very poor at alerting on novel intrusion methods History of Intrusion Detection Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

9. History of ID Alert Correlation and Data Fusion Data Fusion Techniques Visualizations Part 1: The Current State of Technology in ID Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

10. Correlate by Source, Destination or Attack Method Non-Trivial port-number vs. service name, IP address vs. hostname, etc. (Cuppens 2001) Need Adaptors – Different systems not designed for fusion (Debar et al. 2001) Promise of better understanding… see next slide Alert Correlation and Data Fusion Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

11. Understanding Through Correlation Adapted from (Debar et al. 2001) Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

12. JDL Fusion Model (Hall and McMullen 2004) Alert Correlation and Data Fusion Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

13. JDL Fusion Model (Hall and McMullen 2004) Alert Correlation and Data Fusion Source Pre- Processing Level 3 Threat Refinement Level 2 Situation Refinement Level 1 Object Refinement Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

14. History of ID Alert Correlation and Data Fusion Data Fusion Techniques Visualization of Underlying and Fused Data Part 1: The Current State of Technology in ID Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

15. Bayesian Inference Complete list of all possible states of the system Probabilities of current state Need for accurate historical data (Holsopple et al. 2006) D-S Theory No need for exact knowledge Sort out independent evidence and combine it using the Dempster Rule Very human-like logical combination Can combine evidence of non-similar sources/data types Data Fusion Techniques Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

16. Data Mining Algorithms Support Vector Machines (SVMs) (Liu et al. 2007 x3) Neural Networks (Wang et al. 2007) May be helpful in rapidly combining multiple sources of similar data Thomas and Balakrishnan (2008) Combined alert data from 3 different IDSs (PHAD, ALAD, Snort) using MLFF-NN Tested vs. DARPA 1999 data set Showed improved detection rates of the known data over each individual IDS (68% vs. 28%, 32%, 51%) Data Fusion Techniques Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

17. History of ID Alert Correlation and Data Fusion Data Fusion Techniques Visualizations Part 1: The Current State of Technology in ID Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

18. Based on Network Topology Based on Geopolitical Topology Network Traffic Representations Alert and Track-Based Displays Visualizations Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

19. Hierarchical Network Map from Mansmann and Vinnik (2006) Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

20. Representation of Threats and Actors on a Geopolitical Map from (Pike et al. 2008) Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

21. Representation of host to port to remote port to remote host of network traffic from (Fink et al. 2004) Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

22. Panel Displaying Network Connections from a Single Host from (Fischer et al. 2008) Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

23. Representing the Three Ws from (Foresti et al. 2007) Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

24. Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

25. Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis Part 2: What are We Trying to Accomplish? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

26. Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis Part 2: What are We Trying to Accomplish? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

27. (Computer) Security is… Manunta (1999) Security is interaction of Asset (A), Protector (P) and Threat (T) in a given Situation (Si) CIA Triad (Tipton et al. 2007) Confidentiality Integrity Availability Bishop (2003) Only authorized actions can be executed by authorized users Definitions… Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

28. Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis Part 2: What are We Trying to Accomplish? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

29. Endsley (1995) State of Knowledge Elements Situation Future Projection “Awareness Machine” unlikely Focus instead on “awareness support technologies” Theory of Situational Awareness Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

30. Endsley (1995) Theory of Situational Awareness Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

31. Mapping of IDS Fusion tasks between JDL Model and Endsley SA Model. From Yang et al. (2009) Higher Levels of Fusion = Situational Awareness Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

32. INFERD Level 2 Fusion Engine – Based on a priori knowledge from system experts – pattern matching attack methods and known vulnerabilities of the system TANDI Level 3 Fusion – Projection of future attacks based on knowledge of vulnerabilities of the system (Yang et al. 2009) Higher Levels of Fusion Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

33. Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis Part 2: What are We Trying to Accomplish? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

34. Sweller et al. (1998) Working Memory (limited capacity) Long Term Memory (unlimited capacity, based on schemas to represent complex, related information) Split Attention Conflicting, Repetitive Modality Effect Cognitive Load Theory Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

35. Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis Part 2: What are We Trying to Accomplish? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

36. Biros and Eppich (2001) – CTA of IDS Analysts in the USAF - 5 capabilities required ID non-local addresses ID source addresses Develop mental image of “normal” behavior Create and maintain SA Knowledge sharing Killcrece et al. (2003) – CTA of gov’t/military security specialists – 3 general categories Reactive Work (majority of the work) Proactive Work Quality Management (training, etc) Cognitive Task Analysis Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

37. D’Amico et al. (2007) – CTA of Network Security Professionals in the Department of Defense Cognitive Task Analysis Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

38. Model Building To understand the contributions of the algorithm builders CTA To understand the needs of the analyst Visualization Recommendations Based on the work above Part 3: Where Do We Go From Here? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

39. Current State of ID History of ID Alert Correlation and Data Fusion Data fusion techniques Visualization of underlying and fused data Theoretical Basis for Understanding SA in the Cyber Security Domain Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis Recommendations for Future Work Model Building - To understand the contributions of the algorithm builders CTA - To understand the needs of the analyst Visualization Recommendations – Based on Needs and Cognitive Capabilities of Analysts Conclusion Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

40. Discussion and Questions Just in case you needed a prompt to ask questions … here it is