1. Multi-Step Attack Defense Operating Point Estimation via Bayesian Modeling under Parameter Uncertainty
Peng Liu, Jun Dai, Xiaoyan Sun, Robert Cole
Penn State University
ARO Cyber Situation Awareness MURI

2. Association & Correlation Multi-Sensory Human Computer Interaction
Cognitive Models & Decision Aids
Instance Based Learning Models
Simulation
Measures of SA & Shared SA
Software
Sensors, probes
Hyper Sentry
Cruiser
Information Aggregation & Fusion
Transaction Graph methods
Damage assessment
Automated
Reasoning
Tools
R-CAST
Plan-based narratives
Graphical models
Uncertainty analysis
Association & Correlation
Data Conditioning
Multi-Sensory Human Computer Interaction
Computer network
Enterprise Model
Activity Logs
IDS reports
Vulnerabilities
Real World
System Analysts
Computer network
Test-bed
ARO Cyber Situation Awareness MURI

3. System Architecture – Cyber Security Perspective
ARO Cyber Situation Awareness MURI

4. ARO Cyber Situation Awareness MURI
Year 4 projects
Multi-Step Attack Defense Operating Point Estimation via Bayesian Modeling
-- PhD Dissertation
Snake: Discover and Profile Network Service Dependencies via network wide SCDGs
-- Tool & paper (in progress)
Patrol: Zero-day attack path detection via network-wide SCDGs
-- ESORICS’13
-- Tool
Cross-layer Bayesian networks to manage uncertainty in cyber SA
-- Paper (in progress)
CLR: Automated recovery plan generation
-- ICICS’13
ARO Cyber Situation Awareness MURI

5. ARO Cyber Situation Awareness MURI
Year 4 accomplishments
Publications:
-- 1 PhD dissertation
-- 5 journal papers
-- 11 conference papers
-- 1 book chapter
Tools:
-- Patrol
-- Snake (in progress)
Tech transfer:
DoD SBIR 12.3 Phase I OSD12-IA5 project “An Integrated Threat feed Aggregation, Analysis, and Visualization (TAAV) Tool for Cyber Situational Awareness,” funded, led by Intelligent Automation, Inc.
Students:
-- Jun Dai (50%), PhD
-- Xiaoyan Sun (50%), PhD
-- Robert Cole (0%), PhD
ARO Cyber Situation Awareness MURI

6. ARO Cyber Situation Awareness MURI
Research Highlight:
Multi-step attack defense operating point estimation via Bayesian modeling
ARO Cyber Situation Awareness MURI

7. ARO Cyber Situation Awareness MURI
Motivation
No real world IDS system is perfect.
-- When an IDS system is configured to achieve a higher true positive rate, usually it would suffer from a higher false positive rate
Such a (true positive rate, false positive rate) tradeoff is called an operating point of the IDS.
The cyber operator can keep tuning the IDS until the estimated operating point is close enough to the desired operating point.
ARO Cyber Situation Awareness MURI 7

8. ARO Cyber Situation Awareness MURI
Problem Statement
Due to the inherent uncertainty associated with gaining cyber SA, operating point estimation won’t be 100% accurate.
Although the estimation problem for individual exploits has been studied in the literature, the estimation problem for multi-step attacks (a chain of exploits) under model parameter uncertainty has not yet been studied.
-- Traditional IDS systems do not explicitly consider uncertainty
ARO Cyber Situation Awareness MURI 8

9. ARO Cyber Situation Awareness MURI
Innovation Claim
We developed the first quantitative multi- step intrusion detection system operating point estimation framework based on Bayesian modeling.
ARO Cyber Situation Awareness MURI 9

10. ARO Cyber Situation Awareness MURI
Approach
Do generalized alert correlation analysis.
Instead of requiring (certain types of) attribute value match (e.g., the destination IP address of one alert matches the source IP of another) between two IDS alerts, we model the rationale for such matches using conditional probabilities and a Bayesian net.
--Similar modeling is used in the ACSAC’04 work by Ning group for a different purpose.
They want to infer unknown intrusion evidence; in contrast, we want to quantify the uncertainty in operating point estimation.
ARO Cyber Situation Awareness MURI 10

11. Research Contribution 1
We developed a novel Bayesian operating point estimation model:
-- General multi-step attack strategies can be precisely specified as a “query” against the model which corresponds to a specific Bayesian network.
-- Our model can propagate parameter uncertainty through the model to a query result.
ARO Cyber Situation Awareness MURI 11

12. Research Contribution 2
Shift from per-exploit detection to per- chain:
In the case of zero parameter uncertainty, we developed an efficient algorithm to enumerate useful operating points within the 2-dimensional design space of:
[detection rate vs. false positive rate]
ARO Cyber Situation Awareness MURI 12

13. Research Contribution 3
For the uncertain parameter case, we studied the special case of serial order multi-step attacks.
We theoretically proved that there exist specific cases under which model parameter uncertainty won’t produce output uncertainty.
ARO Cyber Situation Awareness MURI 13

14. Research Contribution 4
We found that operating points could become 2- dimensional operating boxes.
The general problem of operating box enumeration is highly computationally complex. We conducted experiments evaluating two heuristic solutions.
Experimental results show a heuristic solution (our operating point enumeration algorithm) provides results very close to full enumeration.
Results show the significance of uncertainty in the multi-step attack detection cases considered.
ARO Cyber Situation Awareness MURI 14

15. ARO Cyber Situation Awareness MURI
Year 5
Snake: Discover and Profile Network Service Dependencies via network wide SCDGs
-- Tool & paper (in progress)
Joint project with NIST: Cloud-wide vulnerability analysis
-- In progress
Joint project with NEC Labs: System-call-level security intelligence
-- In progress
Cross-layer Bayesian networks to manage uncertainty in cyber SA
-- In progress
Tool integration: with GMU, NCSU, etc.
-- In progress
ARO Cyber Situation Awareness MURI

16. Objectives: Improve Cyber SA through:
ARO MURI: Computer-aided Human-Centric Cyber Situation Awareness: SKRM Inspired Cyber SA Analytics
Penn State University (Peng Liu)
Tel ,
Objectives: Improve Cyber SA through:
A Situation Knowledge Reference Model (SKRM)
A systematic framework for uncertainty management
Cross-knowledge-abstraction-layer SA analytics
Game theoretic SA analytics
DoD Benefit:
Innovative SA analytics lead to improved capabilities in gaining cyber SA.
Uncertainty analysis
Scientific/Technical Approach
Leverage knowledge of “us”
Cross-abstraction-layer situation knowledge
integration
Network-wide system all dependency analysis
Probabilistic graphic models
Game theoretic analysis
Accomplishments
A suite of SKRM inspired SA analytics
A Bayesian Networks approach to uncertainty
A method to identify zero-day attack paths
A signaling game approach to analyze cyber
attack-defense dynamics
Challenges
Systematic evaluation & validation
ARO Cyber Situation Awareness MURI

17. ARO Cyber Situation Awareness MURI
Q & A
Thank you.
ARO Cyber Situation Awareness MURI